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Preface 



Washington State has implemented an innovative policy that uses intermediate 
perfonnance measures and incentive funding to encourage the state’s community and 
technical colleges to adopt practices that increase rates of student progression and 
completion. Under this policy, called the Student Achievement Initiative (SAI), colleges 
earn points when students achieve one or more educational milestones, or -achievement 
points,” which are organized along a continuum from remedial programs (which include 
adult basic education and pre-college -developmental” education) through the completion 
of credentials and training programs. 

In this report, Clive Belfield, an economist at Queens College-CUNY and a 
research affiliate at CCRC, examines the recent perfonnance of Washington State two- 
year colleges under this policy. Belfield analyzes the patterns and detenninants of 
achievement points earned by colleges since the policy was enacted in 2007. This report 
extends two similar analyses Belfield conducted in 2009 and 2011. The current analysis 
takes advantage of data from a longer period to address the following questions, which 
were fonnulated in consultation with staff of the Washington State Board for Community 
and Technical Colleges and with input from the Education Committee of the Washington 
Association of Community and Technical Colleges. 

1. Do patterns observed in the initial quantitative analysis (e.g., 
the importance of college size and the growing dominance of 
basic skills and pre-college measures) persist into 2011? Have 
new patterns emerged? How do these patterns affect the 
allocation of funding? Are these patterns consistent with the 
SAI measurement and funding principles? 

2. Do alternative performance measures yield different funding 
allocations? For example, what would happen if colleges were 
rewarded based on changes in points per student rather than 
changes in total points? 

3. Is there evidence that student momentum has increased since 
the SAI was established — that is, are there more students that 
are progressing further toward completion of postsecondary 
credentials? 
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Belfield’s findings on these questions are intended to inform an ongoing review of 
the SAI policy being conducted by the Washington Association of Community and 
Technical Colleges. His analysis is also part of a larger three-year evaluation of the SAI 
being conducted by CCRC and the Institute for Higher Education Leadership and Policy 
(IHELP) with funding from the Bill & Melinda Gates Foundation. The larger study 
examines the effects of the SAI on efforts by Washington two-year colleges to implement 
policies and practices intended to improve student outcomes; it also seeks to capture 
lessons for other states seeking to improve student outcomes and college perfonnance 
through state policy. 1 This spring, CCRC and IHELP researchers will conduct site visits 
and interviews at more than half of the colleges in the Washington State system to 
observe the effects of the SAI and examine how the practices of colleges influence the 
numbers Belfield presents here. 



Davis Jenkins 
CCRC 
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Early lessons for other states were presented in a policy paper by Shulock and Jenkins (2011). 
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Executive Summary 



This paper extends an earlier analysis of Washington State’s Student 
Achievement Initiative (SAI). The goal of the initiative is to increase academic 
achievement at the state’s 34 community and technical colleges. Under the SAI, since 
2009, the state has awarded some funds based on how well colleges have increased 
student achievement relative to their own prior perfonnance. Earlier analyses (Bel Held 
2009, 2011) indicated that the SAI was operating as intended. This paper examines how 
college perfonnance has changed over the period from the baseline in 2007 to 201 1, 
using an additional two years of data beyond what was used for the earlier analyses. 

The SAI measures students’ academic achievement in points, using six metrics: 
improvement in performance on assessment of basic skills; advancement across levels of 
developmental education toward college readiness; accumulation of 15 college credits; 
accumulation of 30 credits; completion of quantitative reasoning courses; and degree (or 
certificate or apprenticeship) completion. The measures used in the current analysis 
include total points, the change in total points, points generated for each individual 
metric, and points per student generated by each college. The following are the main 
findings of the study. 

Patterns and Determinants of Changes in Achievement Points 

Change in total points and points per student: 

• In the baseline year, the average point total per college was 
8,684. By 2009, the average rose to 10,365; by 2010, it rose to 
1 1,598; and by 201 1, it dropped slightly to 1 1,465. 

• Over the period from baseline to 201 1, the average college 
increased its point total by 3 1 percent. 

• During this period, points per student increased by 29 percent. 

Enrollment was flat, but more students enrolled more 
intensively; full-time equivalent enrollment grew by 4 1 percent. 

Therefore, points per full-time equivalent fell by 4 percent. 
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Influence of college size: 

• College enrollment size and total points were correlated. 
Enrollment size became modestly more influential over time. A 
college with 1 ,000 more students gained on average 60 more 
points per year. This is consistent with the original design of the 
SAI; larger colleges were expected to be able to obtain bigger 
awards on the premise that they would require more resources 
to make changes. 

Influence of basic skills and college readiness points: 

• The relative importance of the six metrics balanced out over 
time. The early trend — in which basic skills and college 
readiness points grew disproportionately — was not sustained 
into 2011. 

• Consistent with the SAI’s aim to encourage colleges to serve all 
students, including those from disadvantaged backgrounds, the 
basic skills metric appears to have encouraged enrollment from 
traditionally underserved groups. 

Effects of student characteristics: 

• There is no simple way to predict points accumulation for a 
given college or type of student. The characteristics of a 
college’s students — such as age, full-time enrollment status, 
race, or prior education — do not strongly influence total points 
or the change in points. Although there were some significant 
influences on points across each of the six metrics, they do not 
result in large effects when the data are viewed in aggregate, 
and they depend on whether the student is full-time or part-time. 

• There is little evidence that colleges serving more at-risk, low- 
income students are penalized by the SAI awards method. This 
is consistent with one of the key SAI design principles — points 
accumulation is not supposed to be driven by the types of 
students enrolled. 

Comparison with Alternative Performance Measures 

• The SAI fonnula for points results in little volatility in 
perfonnance by individual colleges over time. Colleges that 
gained a lot of points from the 2007 baseline to 2009 also 
gained a lot of points from baseline to 201 1. 
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• Alternative funding methods — such as ranking colleges based 
on changes in points for FTE — would yield different results in 
terms of which colleges would receive the most funds. 

Evidence of Increase in Student Momentum 

• Rates of progression or momentum were similar for students in 
college-level transfer and workforce programs. Interestingly, 
the momentum of students in either transfer or workforce 
programs did not seem to be dependent on whether they first 
took pre-college remedial courses or entered directly into 
college-level programs. 

• Few students who earned basic skills points accumulated 
additional college-level points over the following three years. 

• Some modest evidence suggests that momentum improved 
since baseline, primarily among students in college-level 
courses who were accumulating credits and making progress 
toward completion. 

• About half of students did not accumulate any points. These 
students and basic skills students comprise a large proportion of 
all students. Improvements in momentum by the smaller 
number of students who did make progress had a small effect on 
points accumulation overall. 

• Colleges may be more likely to improve student momentum if 
they focus on the gains that can be made among students when 
they first enter the institution. 
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1. Introduction 



The Student Achievement Initiative (SAI) is a performance funding system for all 
public two-year colleges in Washington State. - It awards funding based on how well 
colleges increase academic achievement with the goal of helping students progress 
further in their college careers. Achievement is measured in points, using six metrics: 
improvement in performance on assessment of basic skills; advancement across levels of 
developmental education toward college readiness; accumulation of 15 college credits; 
accumulation of 30 college credits; completion of quantitative reasoning courses; and 

•3 

degree (or certificate or apprenticeship) completion/ A college is deemed to be raising 
achievement levels if its point total increases relative to its own past performance. 

Successful performance funding systems have clear perfonnance goals and tie 
funding directly to performance (Dougherty & Natow, 2010). By focusing on 
achievement and by explicitly articulating how points are awarded, the SAI does both of 
these things. Perfonnance funding systems must also be robust, in the sense that awards 
are not sensitive to trivial differences in how they are calculated. However, they must 
allow for flexibility so that colleges can change their practices to accumulate more points. 
Earlier analyses found evidence that the SAI was broadly satisfying these two criteria as 
well (Belfield, 2009, 2011). Colleges were accumulating points at a rate faster than their 
enrollment growth, mostly with more basic skills and college readiness points. There was 
no evidence that colleges were disadvantaged by the composition of their student body. It 
was not possible to identify student characteristics that allowed colleges to earn points 
more easily than others, and there was no evidence that serving at-risk, low-income 
students prevented colleges from accumulating points. 

This report updates and extends these earlier analyses using data collected from 
2007 to 201 1. Following the earlier studies, this paper presents a detailed empirical 
analysis of how many points colleges accumulated (both in total and by metric), as well 

2 For details on the SAI, see: www.sbctc.ctc.edu/college/e studentachievement.aspx . See also Jenkins, 
Ellwein, and Boswell (2009). 

3 These metrics encompass the full range of Washington State community and technical college mission 
areas, including adult basic skills and developmental education as well as baccalaureate transfer and career- 
technical education. That said, the State Board for Community and Technical Colleges made an explicit 
policy decision to award points for student achievement in basic skills and developmental programs in 
order to encourage colleges to serve more academically unprepared students and thus help address the 
growth in population of educationally disadvantaged adults in the state. 
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as which college-level and student characteristics were associated with points 
accumulation. Updating the earlier analyses allows for the validation of the initial 
findings and an investigation of whether initial trends were sustained over time. The 
current paper also re-examines the question of whether different measures of institutional 
perfonnance might yield different funding allocations. New to this report is an analysis of 
student momentum — i.e., whether students began making further progress after the SAI 
was introduced. The system was set up so that colleges could generate points by serving 
more students or improving the rates at which students as progress through college. 
Earning points in the latter way suggests that colleges are serving students more 
effectively. In previous analyses, progress toward this goal could not be adequately 
examined because only one round of data was available. With new data from 2007 to 
201 1, it is possible to map the pathways of individual students over time (Leinbach & 
Jenkins, 2008). These pathways are described in terms of points, with progression 
defined as reaching a higher level of points. 

This analysis, in conjunction with information on how colleges have implemented 
the SAI (see Shulock & Jenkins, 2011), provides additional evidence on the validity of 
the SAI as a performance funding system. This analysis should not be interpreted as a full 
evaluation of the SAI. It is focused on patterns of achievement points accumulation by 
colleges and their students. The results should be understood in light of the intended 
goals of the SAI and in conjunction with field research at individual colleges being 
conducted by the Community College Research Center and the Institute for Higher 
Education Leadership and Policy. 

The dataset used in this study includes all students enrolled in Washington State’s 
34 community and technical colleges, with infonnation on students’ demographic 
characteristics, prior education levels, and SAI points achieved by term. Data for 
individuals are combined to allow for analysis at the college level. 4 Additional college- 



4 Seattle Vocational Institute data are subsumed into data for Seattle Central. Results are reported for 
individual colleges even though some colleges are directly linked to each other. For example, Spokane 
Falls and Spokane Community College are part of the same district and share resources. Spokane Falls data 
include the Institute for Extended Learning basic skills center, with Spokane Community College serving 
students further along in their college path. Allocations and rewards are awarded to the district. Hence, the 
resource implications of their respective point totals are not independent for these two colleges. 
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specific data from the Integrated Postsecondary Education Data System (IPEDS) is 
integrated into the analysis. 

The baseline year for the SAI is the academic year (AY) of 2007; this study uses 
data that were collected from 2007 through 2011. Since establishing the baseline, the SAI 
has begun rewarding colleges for increases in their total points. AY 2009 was the first full 
year of the SAI; 2008 was a preparatory/leaming year, in which funding was not 
contingent on points. Performance funding in this first round was awarded based on 
points accumulated over the first two years, i.e., with 2007 as the baseline until 2009; in 
subsequent rounds, it depended on annual points accumulation. Therefore, the current 
analysis looks at trends in two dimensions. One is the annualized trend, to see how 
colleges perform year-on-year. 5 The second is the longer-tenn changes from baseline to 
201 1, to see if the system has created performance gaps among colleges. Identifying 
these trends is helpful for policymakers looking at the short-term impacts of the SAI and 
how colleges have adjusted since the baseline year. 6 



2. Absolute Points Per College 
2,1 Total Points Per College 

The SAI funding fonnula rewards colleges as their absolute total achievement 
points increase year-on- year. The change in total points from all six metrics detennines 
the incremental funding awarded to each college. This fonnula was designed such that 
colleges would get more points if they enrolled more students who satisfied a metric. In 
part, this design decision was based on the principle that efforts to improve student 
success would require more resources at larger colleges. 

Total points per college per year are shown in Table 1. In the baseline year, the 
average point total was 8,684 points. In 2009, the average rose to 10,365; in 2010, it rose 
to 1 1,598 points; and in 201 1, it dropped slightly to 1 1,465. On average, across all 

5 This study uses four -years” of data (2007-08, 2009, 2010, and 2011), which are treated as consecutive. 
The final year (2011) is based on data as of April 17 2012. Revisions in the data are such that there are 
discrepancies of a few points per college. 

6 It is not possible to fully isolate those changes resulting from the SAI from any changes are due to broader 
economic conditions (e.g., the Great Recession). 
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colleges, annual point gains were +19 percent (2007-09), +12 percent in 2010, and -1 
percent in 201 1. The gain in points from baseline to 201 1 was +31 percent. Since 
baseline, the college with the biggest change in points was College 007 (+7,556); the 
college that gained the least was College 027 (+287 points). So, although several colleges 
lost points in any given year, no college has lost points in total since the baseline year. 



Table 1 

Total Points Per College 



College 


Baseline 
AY 2007 


AY 2009 


AY 2010 


AY 2011 


% Change 
07-09 


% Change 
09-10 


% Change 
10-11 


% Change 
07-11 


College 001 


5,710 


6,781 


8,507 


6,989 


19 


25 


-18 


22 


College 002 


16,103 


18,722 


20,975 


21,983 


16 


12 


5 


37 


College 003 


2,846 


4,383 


4,970 


5,060 


54 


13 


2 


78 


College 004 


4,164 


5,303 


5,513 


5,231 


27 


4 


-5 


26 


College 005 


4,065 


4,898 


5,566 


5,488 


20 


14 


-1 


35 


College 006 


5,754 


6,005 


6,370 


6,565 


4 


6 


3 


14 


College 007 


15,874 


19,344 


24,385 


23,430 


22 


26 


-4 


48 


College 008 


5,652 


7,470 


9,256 


8,974 


32 


24 


-3 


59 


College 009 


11,060 


13,849 


12,228 


12,605 


25 


-12 


3 


14 


College 010 


13,650 


17,312 


18,666 


18,819 


27 


8 


1 


38 


College Oil 


13,267 


16,246 


19,135 


18,134 


22 


18 


-5 


37 


College 012 


4,251 


4,531 


5,073 


4,627 


7 


12 


-9 


9 


College 013 


12,244 


14,595 


15,986 


16,159 


19 


10 


1 


32 


College 014 


11,996 


15,648 


19,369 


18,121 


30 


24 


-6 


51 


College 015 


6,950 


8,130 


9,108 


8,944 


17 


12 


-2 


29 


College 016 


5,021 


6,675 


8,856 


8,350 


33 


33 


-6 


66 


College 017 


10,073 


11,879 


13,818 


14,216 


18 


16 


6 


46 


College 018 


4,384 


6,020 


5,781 


5,948 


37 


-4 


3 


36 


College 019 


5,376 


6,860 


7,241 


7,372 


28 


6 


2 


37 


College 020 


11,445 


12,057 


13,130 


13,785 


5 


9 


5 


20 


College 021 


8,498 


9,542 


10,551 


10,611 


12 


11 


1 


25 


College 022 


12,153 


13,619 


16,358 


15,847 


12 


20 


-3 


30 


College 023 


8,355 


9,329 


9,690 


10,004 


12 


4 


3 


20 


College 024 


8,133 


10,437 


11,805 


10,257 


28 


13 


-13 


26 


College 025 


8,547 


9,792 


10,527 


9,630 


15 


8 


-9 


13 


College 026 


7,631 


8,558 


9,557 


10,707 


12 


12 


12 


40 


College 027 


9,845 


11,158 


11,197 


10,132 


13 


0 


-10 


3 


College 028 


14,794 


18,959 


19,024 


17,926 


28 


0 


-6 


21 


College 029 


8,049 


9,453 


10,665 


10,709 


17 


13 


0 


33 


College 030 


9,801 


11,967 


13,498 


15,121 


22 


13 


12 


54 


College 031 


6,018 


6,657 


7,682 


7,778 


11 


15 


1 


29 


College 032 


6,315 


6,473 


6,529 


6,722 


3 


1 


3 


6 


College 033 


8,017 


8,825 


10,552 


10,876 


10 


20 


3 


36 


College 034 


9,218 


10,942 


12,777 


12,684 


19 


17 


-1 


38 


Average per college 


8,684 


10,365 


11,598 


11,465 


19 


12 


-1 


31 
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Table 1 also shows the percentage change in points per college by year. 7 Within 
any given year, there was significant variation in growth rates. For example, in 2007-09, 
the total points generated by College 003 grew by considerably more than the point totals 
for any other college (+54 percent versus the next highest of +37 percent); four colleges 
grew by less than 10 percent. Over time, the gaps in point growth between colleges 
widened. From baseline to 201 1, the point total for College 003, which had the highest 
growth rate, increased by 78 percent; all colleges gained points, but three gained fewer 
than 10 percent. The relative positions of the colleges remained stable. Colleges with 
high totals in the baseline year also had high totals in 201 1; this was a strong relationship, 
with a Pearson correlation coefficient of 0.96. 

Earlier studies (Belfield 2009, 2011) suggested that the spread of total points was 
narrowing. This trend was not sustained through the period for the current study. In 2007, 
College 002, which had the most points, had 5.7 times as many total points as College 
003, which had the fewest; by 2009, the ratio of most to least points fell to 4.4. By 201 1, 
this ratio rose again to 5. 1 . Yet, there was no general compression in the spread of total 
points per college; the coefficient of variation (standard deviation divided by the mean) 
was almost identical in all three years (at 0.41-0.44). Any divergence in points and points 
growth took place primarily at the extremes, involving a few colleges. Even this 
interpretation is tentative in light of the 2011 data: Annual growth in points in 201 1 was 
-1 percent, down sharply from the year before (+12 percent). This negative growth 
slightly compressed the point differences across colleges. 

Unsurprisingly, larger colleges consistently earned more total points. A similar 
relationship exists between total college expenditures and total points in 201 1, as shown 
in Figure 1. Broadly, total points increased in a linear fashion with expenditures, although 
the association weakens as college expenditures grow. 



7 The calculations are based on averages per college, not the overall total of points across all 34 state 
colleges. 



10 




Figure 1 

Total Expenditures and Total Points Per College in 2011 




2.2 Points in Each Metric Per College 

Total points are the sum of points accumulated across each of the six metrics. 

This section examines which metrics yield the most points. This examination also reveals 
whether the relative points between metrics differ across colleges (e.g., some colleges 
may have mostly basic skills points, others may have mostly college readiness points). 

Figure 2 shows how SAI points are distributed across each metric. At baseline, 
colleges were accumulating approximately two thirds of their points evenly across three 
metrics: basic skills, college readiness, and 15 credits. The remaining points were 
accumulated via the other three metrics: 30 credits, quantitative reasoning, and degree 
completion. These proportions have not changed; by 201 1, the proportion of points from 
each metric was almost exactly the same as in 2007. Until 2010, there appeared to be a 
trend toward increases in basic skills and college readiness points, but this was not 
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sustained into 2011. Table 2 reports the points by metric as averages per college. In the 
baseline year, the average college had 2,087 points from basic skills; by 201 1, the 
average rose 38 percent to 2,872. At the same time, points from all other metrics 
increased. In fact, the average college saw college readiness and degree completion 
points rise the fastest between baseline and 2011, at +46 percent and +42 percent 
respectively. This suggests that achievement has risen. The slight decrease in total points 
between 2010 and 2011 was almost entirely a result of a decrease in basic skills points. 

However, this broad analysis masks within-college variations. For each metric, 
some colleges lost points and others experienced significant growth (e.g., of 50 percent 
per year). In any given year, one third of colleges lost points in basic skills, 15 credits, 
and quantitative reasoning; for the other metrics, one in five colleges lost points in any 
given year. Thus, colleges could easily lose points in each category in a particular year. 



Figure 2 

Points by Metric Over Time 
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8 It is possible that this initial increase in basic skills points reflected improved accounting methods. 
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Table 2 

Points by Metric: Average Per College 



Metric 


Baseline 
AY 2007 


AY 2009 


AY 2010 


AY 2011 


% Change 
07-09 


% Change 
09-10 


% Change 
10-11 


% Change 
07-11 


Basic skills 


2,087 


2,788 


3,183 


2,872 


34 


14 


-10 


38 


College readiness 


1,811 


2,166 


2,580 


2,641 


20 


19 


2 


46 


15 credits 


1,777 


2,063 


2,172 


2,100 


16 


5 


-3 


18 


30 credits 


1,335 


1,538 


1,680 


1,662 


15 


9 


-1 


24 


Quantitative reasoning 


1,000 


1,059 


1,161 


1,229 


6 


10 


6 


23 


Degree completion 


674 


751 


822 


961 


11 


9 


17 


43 


Average per college 


8,684 


10,365 


11,598 


11,465 


19 


12 


-1 


31 



Table 3 illustrates the college-level volatility across each metric from baseline to 
2011. Overall, there appear to be more opportunities for colleges to earn basic skills and 
pre-college points. For basic skills, some colleges had significant growth in points 
(College 003, +128 percent; College 014, +100 percent), and one had significant losses 
(College 032, -55 percent). For college readiness, three colleges grew extremely quickly 
(College 003, +315 percent; College 021, +125 percent; and College 008, +117 percent). 
For quantitative reasoning, three colleges grew substantially (College 018, +137 percent; 
College 001, +125 percent; and College 016, + 106 percent). In contrast, the growth in 
points for the accumulation of 15 or 30 college credits was less dramatic, only changing 
by +50 percent at a few colleges. This disparity makes sense because the basic skills and 
college readiness metrics require students to improve their scores on a basic skills test or 
progress through a sequence of remedial courses rather than take and pass a college-level 
math course or accumulate college credits that count toward completion of a credential. A 
few colleges did increase their degree completion points substantially (College 008, +115 
percent; College 024, +107 percent; and College 015, +106 percent). On this metric, only 
one college accumulated fewer points in 201 1 than at baseline (College 020, -4 percent). 

Overall, these trends have led to shifts in how some colleges accumulate points, 
though the ultimate effect is not dramatic, even at the tails of the distribution. Colleges 
that grew faster tended to start from a lower base, so they converged to the average. 
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Table 3 

Points by Metric Per College: Percentage Change from Baseline to 2011 



College 


Basic Skills 


College 

Readiness 


15 Credits 


30 Credits 


Quantitative 

Reasoning 


Degree 

Completion 


College 001 


76 


26 


-1 


3 


125 


42 


College 002 


66 


32 


28 


35 


36 


29 


College 003 


128 


315 


28 


38 


86 


36 


College 004 


20 


64 


3 


20 


10 


20 


College 005 


0 


40 


39 


48 


51 


48 


College 006 


-5 


39 


4 


18 


28 


46 


College 007 


45 


55 


49 


54 


12 


64 


College 008 


44 


117 


38 


23 


53 


115 


College 009 


3 


29 


12 


15 


0 


33 


College 010 


45 


52 


24 


32 


7 


69 


College Oil 


22 


45 


43 


43 


52 


24 


College 012 


-28 


78 


13 


25 


-14 


46 


College 013 


27 


59 


13 


15 


17 


99 


College 014 


100 


34 


13 


14 


61 


20 


College 015 


20 


38 


5 


18 


56 


106 


College 016 


65 


83 


28 


49 


106 


99 


College 017 


29 


53 


26 


30 


57 


65 


College 018 


40 


74 


-11 


20 


137 


79 


College 019 


61 


59 


24 


29 


30 


14 


College 020 


5 


71 


12 


22 


-2 


-4 


College 021 


22 


125 


29 


28 


-6 


46 


College 022 


95 


38 


-2 


3 


-15 


14 


College 023 


10 


50 


6 


32 


5 


59 


College 024 


60 


40 


-15 


0 


12 


107 


College 025 


17 


11 


9 


5 


8 


39 


College 026 


77 


37 


27 


31 


51 


24 


College 027 




12 


-2 


7 


-23 


21 


College 028 


57 


-13 


10 


12 


0 


22 


College 029 


17 


49 


17 


35 


47 


48 


College 030 


38 


84 


33 


39 


65 


35 


College 031 


10 


49 


24 


23 


32 


59 


College 032 


-55 


40 


23 


27 


18 


29 


College 033 


-6 


41 


54 


53 


31 


21 


College 034 


87 


28 


21 


15 


-6 


2 


Average 


38 


46 


18 


24 


23 


43 
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2.3 Determinants of Points 



This section investigates which characteristics of the colleges determined total 
points, controlling for the influences of other factors. Across the 34 colleges, total points, 
points by metric, and absolute changes in points are regressed against characteristics of 
the student body. 9 At the aggregate level, this investigation reveals whether there are 
systematic influences that may favor colleges that enroll certain types of students. 

The estimates for 2009, 2010, and 2011 are given in Tables A.1-A.3 (located in 
the appendix). Total points per college was strongly related to size of enrollment, or 
headcount. This relationship holds when controlling for the composition of the student 
body as regards socioeconomic status, age, full-time/part-time enrollment, disability, 
race, and prior education. The points-headcount relationship was reasonably consistent 
across all three years. If an average-sized college increased its enrollment by 1 ,000, it 
would have increased its total points by 670-710. In a previous analysis (Belfield, 2011), 
I speculated that the influence of college size might be growing over time; the additional 
years of data demonstrate this to be the case, but the trend is not dramatic. Few other 
student body characteristics were positively associated with total points in a consistent 
way. 10 Thus, point totals were not driven by characteristics of the colleges, such as the 
composition of the student body. This conclusion holds generally for each of the six 
metrics. 11 

Table A.4 shows the change in points across each year and from baseline to 201 1. 
Change in total points from each college’s baseline was measured by which rewards are 
allocated under the SAI policy. Again, the only consistent association was with 



9 All variables are measured contemporaneously. Data for Table A.l is based on the student composition in 

2009. Data for Table A.3 is based on the student composition in 2011. 

10 For example, colleges with more students without a high school diploma or GED had more points in 

2010, but this association is not found in 2009 or 2011. In earlier studies, no correlation was found between 
points and spending per full-time equivalent student (Belfield, 2011). 

11 Tables A.l, A.2, and A.3 also show patterns for each of the six metrics. Again, total points for each 
metric are influenced by the enrollment size of the college. For basic skills, higher point totals are 
associated with colleges with greater representations of Flispanic students and dropouts; this makes sense, 
given that basic skills programs are designed to serve students seeking to improve their facility in English 
and students seeking a high school equivalency credential. College readiness points are not associated with 
any student body characteristics. For the other four metrics, as for the total, there are no clear, strong 
patterns. The patterns that are evident appear plausible — credit accumulation and completion rates are 
higher where more students attend school full-time and where there are fewer students of color — although 
few associations are statistically significant. 
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enrollment size: Bigger colleges accumulated more points. A college with 1,000 more 
enrollments at baseline would have accumulated 165 points more than the average 
college by 201 1. This finding reflects a design principle of the SAI: Bigger colleges were 
expected to obtain bigger points gains. Given that the average college accumulated 
approximately 2,500 more points over the period, this magnitude of this effect is not 
great. The only other variable associated with points gain from baseline to 201 1 was the 
education levels of entering students. Where there were more high school dropouts, 
colleges found it easier to accumulate points. This is probably because serving more 
dropouts enhances a college’s ability to generate basic skills points. 

Figures 3 and 4 show how point totals changed over time. Figure 3 shows the 
point totals at each college in 2007 and in 201 1. The association is strongly linear, and 
there are no outlier colleges. Figure 4 shows gains in points. Generally, colleges that 
gained a lot of points from baseline to 2009 also gained a lot of points from baseline to 
2011. These colleges sustained their early gains and did not revert back to the average by 
2011. Nevertheless, the divergence is not especially strong; the bulk of colleges are 
clustered together along a straight line, with gains of approximately 1,000 points from 
baseline to 2009 and 2,000 from baseline to 201 1. 

Overall, there appears to be no college-level evidence that the points system 
favors schools with particular student compositions either in the short term or the long 
tenn. This, too, was a principle of the SAI: Points accumulation should not be driven by 
the types of students enrolled. 



16 




Points Gain 2007-09 Points Total in 2007 

1000 2000 3000 4000 0 5000 10000 15000 



Figure 3 

Point Totals in 2007 and 2011 at Each College 
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Points Gain 2007-09 and 2007-11 at Each College 
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3. Average Points Per College 



3.1 Points Per Student Per College 

From an economic perspective, it is useful to examine the determinants of points 

as a function of the resources available. More effective colleges might be defined as those 

that generate more points per individual student (even if enrollment falls); more efficient 

1 2 

colleges may be defined as those that generate more points per dollar of expenditure. 

In this paper, headcounts serve as the primary measure of student enrollments, 
although FTEs are also considered. Table 5 shows points per student (PPS) across the 
years 2007-2011. Colleges earned 0.66 points per student in 2007; this number rose to 
0.74 by 2009 (+15 percent) and 0.83 by 2010 (+13 percent), stabilizing in 201 1 (after 
rounding, the change is effectively zero percent). From baseline to 201 1, points per 
student rose by 29 percent on average across the 34 colleges. 

The colleges varied in their growth rates of PPS within any given year. From 
baseline to 2009 and from 2009 to 2010, only a few colleges experienced negative 
growth rates. Between 2010 and 2011, the majority of colleges saw negative growth 
rates. Over the long term, one college (College 021) doubled its PPS; two colleges 
(College 015 and College 027) saw effectively no change in PPS. However, the spread of 
PPS across the colleges has narrowed since baseline. 14 Although there was some 
volatility in PPS in any given year, this did not cause colleges’ PPS to diverge over time. 

The results for FTEs (see Table A. 5) differ somewhat from those obtained using 
student headcounts. Points per FTE (PPFTE) fell in both 2010 and 2011. The baseline- 
201 1 growth in PPFTE was -4 percent, compared to +29 percent for PPS. Thus, the 
number of FTEs rose faster than the total number of students. However, the rankings of 
colleges obtained using PPFTE and PPS are similar, although the latter measure shows 
greater stability across the years (Pearson correlation coefficient > 0.88). 15 Comparisons 



12 Under the SAI award scheme, a college whose enrollment decreases in a given year can still earn points 
by increasing points per student. 

13 The headcount measure weights part-time students equally with full-time students even though colleges 
vary in their proportions of part-time students enrolled. Empirically, these enrollment measures 
(headcounts and FTEs) are strongly correlated. 

14 The coefficient of variation has fallen from 0.24 to 0.17 and the high/low ratio from 2.8 to 2.0. 

15 Changes in enrollment and points were such that the PPFTE spread fell in the first year (2007-09) but 
rose subsequently. The coefficient of variation was 0.2 but rose to 0.26 by 201 1; the high-low ratio rose 
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based on PPFTE show one college — College 020 — to be much more effective than the 
rest, but this result is not found using PPS. 



Table 5 

Points Per Student 



College 


Baseline 
AY 2007 


AY 2009 


AY 2010 


AY 2011 


% Change 
07-09 


% Change 
09-10 


% Change 
10-11 


% Change 
07-11 


College 001 


0.44 


0.45 


0.62 


0.55 


1 


40 


-12 


24 


College 002 


0.48 


0.58 


0.58 


0.61 


21 


0 


5 


27 


College 003 


0.32 


0.49 


0.57 


0.56 


53 


15 


-1 


75 


College 004 


0.77 


1.06 


1.04 


0.95 


38 


-2 


-9 


23 


College 005 


0.81 


0.96 


1.07 


1.06 


18 


12 


-1 


30 


College 006 


0.72 


0.77 


0.82 


0.86 


7 


6 


6 


20 


College 007 


0.68 


0.85 


0.84 


0.83 


24 


-1 


-1 


21 


College 008 


0.32 


0.48 


0.51 


0.56 


50 


6 


11 


76 


College 009 


0.88 


0.96 


0.95 


0.93 


9 


-1 


-2 


6 


College 010 


0.84 


0.95 


0.99 


0.97 


13 


4 


-2 


15 


College Oil 


0.71 


0.79 


0.90 


0.82 


11 


14 


-8 


16 


College 012 


0.77 


0.78 


0.91 


0.91 


1 


16 


0 


18 


College 013 


0.77 


0.82 


0.87 


0.81 


7 


6 


-7 


6 


College 014 


0.78 


0.83 


0.97 


0.95 


8 


17 


-3 


22 


College 015 


0.87 


0.82 


0.91 


0.87 


-6 


10 


-4 


-1 


College 016 


0.67 


0.74 


0.91 


0.90 


11 


23 


-2 


34 


College 017 


0.70 


0.79 


0.86 


0.87 


13 


10 


1 


25 


College 018 


0.52 


0.67 


0.77 


0.77 


30 


15 


0 


50 


College 019 


0.86 


1.06 


1.07 


1.07 


23 


1 


0 


24 


College 020 


0.59 


0.59 


0.63 


0.69 


0 


8 


8 


17 


College 021 


0.41 


0.63 


0.79 


0.85 


52 


27 


7 


106 


College 022 


0.63 


0.68 


0.76 


0.75 


9 


11 


-1 


20 


College 023 


0.52 


0.57 


0.61 


0.63 


10 


6 


4 


21 


College 024 


0.44 


0.58 


0.71 


0.64 


34 


21 


-9 


48 


College 025 


0.73 


0.77 


0.86 


0.84 


6 


11 


-2 


15 


College 026 


0.56 


0.63 


0.74 


0.85 


13 


17 


15 


52 


College 027 


0.76 


0.71 


0.78 


0.75 


-6 


9 


-4 


-1 


College 028 


0.60 


0.74 


0.78 


0.74 


25 


5 


-5 


24 


College 029 


0.55 


0.56 


0.82 


0.89 


2 


46 


8 


62 


College 030 


0.70 


0.88 


0.94 


0.86 


27 


6 


-8 


24 


College 031 


0.70 


0.65 


0.90 


0.85 


-7 


38 


-6 


21 


College 032 


0.79 


0.80 


0.86 


0.85 


1 


7 


-1 


7 


College 033 


0.68 


0.73 


0.85 


0.89 


8 


16 


5 


31 


College 034 


0.86 


0.87 


1.00 


1.05 


2 


15 


5 


22 


Average per college 


0.66 


0.74 


0.83 


0.82 


15 


13 


0 


29 



from 2.9 to 4.1 by 2011. Therefore, the initial pattern — colleges converging slightly or stable in PPFTE — 
was not sustained. 
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3.2 Determinants of Average Points Per College 



This section investigates the college-level characteristics that were associated 
with the average points per student and the change in points per student for each college. 
These associations indicate how the composition of a college influenced the average 
points accumulated on a per-student basis. Tables 6 and 7 show the determinants of 
points per student for each college in 2010 and 2011 respectively. There were few 
statistically significant and consistent detenninants across the student body. Again, the 
SAI did not appear to favor any particular type of student. 

Comparing colleges with larger and smaller enrollments, there is no evidence of 
economies of scale. Larger colleges had more points but not more points per student. 
Large and small colleges increased points per student at similar rates. 16 Average points 
per student for each of the six metrics are estimated in columns 3-8 of Tables 6 and 7. 
Points varied by college within each metric, and different metrics exhibited different 
patterns. The proportion of male students had no effect on average total points, although 
college readiness points and quantitative reasoning points fell with the proportion of male 
students (a result also found in Belfield [2011]). A younger student body was associated 
with more 15-credit and 30-credit points. This seems plausible because in general, 
younger students are more likely to be seeking a credential. 

Given these null associations, it is no surprise that there were no clear drivers of 
changes in points. Table 8 shows the detenninants of the absolute change in points per 
student across each of the years. Again, there were no strong influences of college 
characteristics (not even college size). 

Overall, the composition of the student body did not appear to have a dramatic 
influence on the average points per student or change in points per student at each college 
in total, across any given metric, or over time. This finding reflects the intent of the SAI, 
which aimed to respect that individual colleges may have different missions and serve 
different populations and not to award points disproportionately for the achievements of 
any given student subgroup. 



16 In fact, Tables 6 and 7 show that larger colleges had fewer points per student (as was found for earlier 
years in Belfield [2011]). 
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Table 6 

Average Points Per Student, 2010 



Average Per Student 



Student 

Characteristics 


Average 
Total Points 


Basic Skills 


College 

Readiness 


15 Credits 


30 Credits 


Quantitative 

Reasoning 


Degree 

Completion 


Financial aid 


-0.286 


-0.044 


-0.011 


-0.179 


-0.089 


-0.060 


0.097 




(0.493) 


(0.582) 


(0.313) 


(0.130) 


(0.122) 


(0.105) 


(0.095) 


SES bottom quintile -0.271 


-0.039 


-0.113 


-0.039 


-0.014 


-0.040 


-0.027 




(0.198) 


(0.233) 


(0.126) 


(0.052) 


(0.049) 


(0.042) 


(0.038) 


Age < 20 


0.224 


-0.378 


0.080 


0.206** 


0.201** 


0.166** 


-0.051 




(0.285) 


(0.337) 


(0.181) 


(0.075) 


(0.070) 


(0.061) 


(0.055) 


Age > 65 


0.426 


1.383 


-0.121 


-0.506** 


-0.290 


0.083 


-0.123 




(0.836) 


(0.986) 


(0.531) 


(0.220) 


(0.206) 


(0.178) 


(0.160) 


Disability 


0.434 


0.759 


-0.306 


-0.007 


0.028 


0.008 


-0.047 




(1.162) 


(1.371) 


(0.738) 


(0.307) 


(0.287) 


(0.248) 


(0.223) 


Full-time 


1.035*** 


0.373 


0.334 


0.121 


0.111 


-0.003 


0.097 




(0.361) 


(0.426) 


(0.229) 


(0.095) 


(0.089) 


(0.077) 


(0.069) 


Male 


-0.345 


0.663* 


-0.775*** 


-0.126 


-0.032 


-0.148** 


0.072 




(0.323) 


(0.381) 


(0.205) 


(0.085) 


(0.080) 


(0.069) 


(0.062) 


Asian 


1.019 


1.299 


-0.213 


-0.287 


-0.146 


0.287* 


0.078 




(0.655) 


(0.773) 


(0.416) 


(0.173) 


(0.162) 


(0.140) 


(0.126) 


Black 


-0.204 


-0.059 


-0.022 


0.125 


0.120 


-0.181 


-0.187 




(0.684) 


(0.808) 


(0.435) 


(0.181) 


(0.169) 


(0.146) 


(0.131) 


Hispanic 


0.417* 


0.933*** 


-0.137 


-0.166*** 


-0.138** 


-0.075 


- 0.000 




(0.215) 


(0.254) 


(0.137) 


(0.057) 


(0.053) 


(0.046) 


(0.041) 


Other race 


1.538* 


1.937* 


-0.067 


-0.145 


-0.192 


0.071 


-0.065 




(0.880) 


(1.038) 


(0.559) 


(0.232) 


(0.217) 


(0.188) 


(0.169) 


HS dropout 


0.773** 


0.591 


0.282 


-0.005 


-0.003 


-0.125* 


0.034 




(0.333) 


(0.392) 


(0.211) 


(0.088) 


(0.082) 


(0.071) 


(0.064) 


HS graduate 


0.371 


-0.684 


0.446* 


0.216** 


0.140 


0.281*** 


-0.028 




(0.360) 


(0.424) 


(0.228) 


(0.095) 


(0.089) 


(0.077) 


(0.069) 


Enrollment 


-0.008*** 


-0.001 


-0.002 


-0.001* 


-0.001** 


-0.002*** 


- 0.000 




(0.002) 


(0.003) 


(0.001) 


(0.001) 


(0.001) 


(0.000) 


(0.000) 


R-squared 


0.864 


0.715 


0.780 


0.781 


0.732 


0.845 


0.639 


N 


34 


34 


34 


34 


34 


34 


34 


Note. Enrollment is in thousands. *p < 0.1. **p < 0.05. ***p < 0.01. 
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Table 7 

Average Points Per Student, 2011 



Student 

Characteristics 


Average 
Total Points 






Average Per Student 






Basic Skills 


College 

Readiness 


15 Credits 


30 Credits 


Quantitative 

Reasoning 


Degree 

Completion 


Financial aid 


0.344 


0.076 


0.290 


-0.118 


-0.005 


0.001 


0.101 




(0.351) 


(0.397) 


(0.267) 


(0.085) 


(0.065) 


(0.061) 


(0.065) 


Age < 20 


0.735** 


-0.466 


0.312 


0.308*** 


0.258*** 


0.376*** 


-0.053 




(0.296) 


(0.335) 


(0.225) 


(0.071) 


(0.055) 


(0.051) 


(0.055) 


Age > 65 


0.468 


1.272 


-0.197 


-0.319* 


-0.087 


0.127 


-0.328** 




(0.765) 


(0.867) 


(0.583) 


(0.184) 


(0.142) 


(0.133) 


(0.142) 


Disability 


0.070 


0.740 


-0.158 


-0.212 


-0.241 


-0.021 


-0.039 




(0.997) 


(1.129) 


(0.759) 


(0.240) 


(0.185) 


(0.173) 


(0.185) 


Full-time 


0.155 


0.124 


0.017 


0.002 


-0.019 


-0.081 


0.112** 




(0.286) 


(0.324) 


(0.218) 


(0.069) 


(0.053) 


(0.050) 


(0.053) 


Male 


-0.165 


0.566 


-0.705*** 


-0.045 


-0.018 


-0.107* 


0.143** 




(0.300) 


(0.340) 


(0.228) 


(0.072) 


(0.055) 


(0.052) 


(0.056) 


Asian 


1.631*** 


1.093* 


0.132 


-0.079 


0.038 


0.322*** 


0.124 




(0.492) 


(0.557) 


(0.374) 


(0.118) 


(0.091) 


(0.085) 


(0.091) 


Black 


-0.516 


0.146 


-0.063 


-0.117 


-0.112 


-0.151 


-0.220** 




(0.519) 


(0.588) 


(0.395) 


(0.125) 


(0.096) 


(0.090) 


(0.096) 


Hispanic 


0.384* 


0.886*** 


-0.183 


-0.112** 


-0.098** 


-0.073** 


-0.037 




(0.199) 


(0.225) 


(0.151) 


(0.048) 


(0.037) 


(0.034) 


(0.037) 


Other race 


0.886 


1.421 


-0.474 


0.057 


0.025 


-0.032 


- 0.111 




(0.847) 


(0.959) 


(0.645) 


(0.204) 


(0.157) 


(0.147) 


(0.157) 


HS dropout 


0.584* 


0.731** 


0.290 


-0.137* 


-0.140** 


-0.206*** 


0.046 




(0.305) 


(0.346) 


(0.232) 


(0.074) 


(0.056) 


(0.053) 


(0.057) 


HS graduate 


0.114 


-0.709* 


0.275 


0.230*** 


0.195*** 


0.194*** 


-0.071 




(0.320) 


(0.363) 


(0.244) 


(0.077) 


(0.059) 


(0.055) 


(0.060) 


Enrollment 


-0.007*** 


- 0.000 


-0.002 


-0.001** 


-0.001*** 


-0.002*** 


- 0.000 




(0.002) 


(0.002) 


(0.002) 


(0.001) 


(0.000) 


(0.000) 


(0.000) 


R-squared 


0.869 


0.727 


0.733 


0.843 


0.868 


0.920 


0.749 


N 


34 


34 


34 


34 


34 


34 


34 



Note. Enrollment is in thousands. *p < 0.1. **p < 0.05. ***p < 0.01. 
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Table 8 

Change in Points Per Student 



Change in Points Per Student 



Student Characteristics 


(1) 

2007-09 


(2) 

2009-10 


(3) 

2010-11 


(4) 

2007-11 


Financial aid 


1.073 


0.174 


0.223 


0.930 




(0.758) 


(0.459) 


(0.319) 


(0.696) 


SES bottom quintile 


-0.089 


0.018 


0.118 


0.184 




(0.240) 


(0.194) 


(0.128) 


(0.220) 


Age under 20 


0.821* 


0.171 


0.027 


0.920** 




(0.414) 


(0.291) 


(0.185) 


(0.380) 


Age 65 over 


-0.322 


-0.121 


1.007* 


0.480 




(1.115) 


(0.918) 


(0.540) 


(1.023) 


Disability 


1.586 


-1.845 


-1.123 


-0.802 




(1.251) 


(1.170) 


(0.752) 


(1.148) 


Fulltime 


-0.969** 


-0.219 


-0.650** 


-1.246*** 




(0.405) 


(0.342) 


(0.234) 


(0.372) 


Male 


0.236 


0.248 


0.090 


0.313 




(0.330) 


(0.270) 


(0.209) 


(0.303) 


Asian 


-0.256 


-0.401 


0.572 


0.307 




(0.761) 


(0.612) 


(0.424) 


(0.698) 


Black 


1.109 


0.049 


-0.629 


0.470 




(0.861) 


(0.694) 


(0.443) 


(0.790) 


Hispanic 


-0.150 


-0.121 


-0.040 


-0.144 




(0.302) 


(0.206) 


(0.139) 


(0.277) 


Other race 


-1.583 


0.894 


0.248 


-0.510 




(0.972) 


(0.649) 


(0.569) 


(0.891) 


HS dropout 


0.526 


0.110 


-0.176 


0.334 




(0.579) 


(0.336) 


(0.215) 


(0.531) 


HS graduate 


-0.662 


-0.471 


0.286 


-0.603 




(0.579) 


(0.369) 


(0.232) 


(0.531) 


Base enrollment 


0.000 


-0.002 


-0.002 


-0.006* 




(0.003) 


(0.002) 


(0.002) 


(0.003) 


R-squared 

Observations 


0.394 

34 


0.409 

34 


0.461 

34 


0.600 

34 



Note. Headcount measure of students. Results are unchanged if points per FTE is used. Standard 
errors in parentheses. Enrollment is in thousands. 

*p < 0.1. **p < 0.05. ***p < 0.01. 
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4. Student-Level Analysis 

This section considers which individual student characteristics were associated 
with points. This disaggregated information may help to identify which students are most 
likely to accumulate points and, therefore, where colleges might channel resources to 
increase student success. In turn, this will reveal to policymakers which types of students 
are likely to be encouraged or discouraged by the new performance funding system and 
how the composition of the college might subsequently change. If the SAI is working as 
intended, student characteristics should have no influence or a very weak influence. 

Student-level detenninants of points for 2011 are estimated in Table 9 (results 
were similar for 2010). Table 9 uses total points per individual student as the dependent 
variable. (Table A. 6 uses a binary variable denoting whether the student accumulated any 
points or not). These equations are estimated separately for full-time and part-time 
students, with and without college-level fixed effects. Whereas two thirds of full-time 
students accumulated points, only one third of part-time students did. The total points per 
student variable is more salient (as it mathematically translates into total points per 
college) than whether individual students accumulated any points. The latter is included 
in the analysis because there may be some student groups for whom it is difficult to 
accumulate points, and policymakers may be interested in which groups those are. 
Overall, similar results are obtained from examining total points and whether or not an 
individual student gained any points. These results are largely unchanged when college- 
level fixed effects — controls for institutional characteristics — are included. 

Students on financial aid tended to accumulate more points. This finding is 
plausible because financial aid is typically contingent on pursuing a credential. Student 
age was also influential, although not in a simple, linear way. Students over 65 were 
much less likely to accumulate points relative to all other age groups, and students under 
20 were much more likely to accumulate points relative to all other age groups. The 
influence of other characteristics depended on whether the student was attending school 
full-time or part-time. 



17 College-level fixed effects are dummy variables added for each college to account for college -specific 
influences on student points. 



24 




Table 9 

Total Points Per Individual Student, Academic Year 2011 





Full-Time Students 


Part-Time Students 


Student Characteristics 


(1) 


(2) 


(3) 


(4) 


Financial aid 


0.105*** 


0.094*** 


0.576*** 


0.578*** 




[0.009] 


[0.009] 


[0.007] 


[0.007] 


Age < 20 


0.720*** 


0.695*** 


0.471*** 


0.458*** 




[0.055] 


[0.056] 


[0.008] 


[0.008] 


Age 20-24 


0.211*** 


0.186*** 


0.306*** 


0.298*** 




[0.055] 


[0.056] 


[0.007] 


[0.007] 


Age 25-29 


0.331*** 


0.305*** 


0.318*** 


0.312*** 




[0.055] 


[0.057] 


[0.008] 


[0.008] 


Age 30-34 


0.389*** 


0.359*** 


0.281*** 


0.275*** 




[0.056] 


[0.057] 


[0.008] 


[0.008] 


Age 34-39 


0.392*** 


0.355*** 


0.275*** 


0.270*** 




[0.057] 


[0.058] 


[0.009] 


[0.009] 


Age 40-44 


0.431*** 


0.390*** 


0.277*** 


0.274*** 




[0.058] 


[0.059] 


[0.009] 


[0.009] 


Age 45-49 


0.450*** 


0.413*** 


0.247*** 


0.247*** 




[0.058] 


[0.059] 


[0.010] 


[0.010] 


Age 50-54 


0.488*** 


0.440*** 


0.161*** 


0.164*** 




[0.060] 


[0.061] 


[0.010] 


[0.010] 


Age 55-59 


0.373*** 


0.326*** 


0.086*** 


0.092*** 




[0.063] 


[0.064] 


[0.011] 


[0.011] 


Age 60-64 


0.343*** 


0.285*** 


0.009 


0.020 




[0.076] 


[0.076] 


[0.012] 


[0.012] 


Disability 


-0.059*** 


-0.064*** 


0.052*** 


0.050*** 




[0.019] 


[0.019] 


[0.011] 


[0.011] 


Male 


-0.077*** 


-0.071*** 


0.019*** 


0.020*** 




[0.009] 


[0.009] 


[0.004] 


[0.004] 


Asian 


0.218*** 


0.262*** 


0.389*** 


0.359*** 




[0.016] 


[0.016] 


[0.008] 


[0.008] 


Black 


-0.099*** 


-0.063*** 


0.270*** 


0.241*** 




[0.017] 


[0.018] 


[0.010] 


[0.010] 


Hispanic 


0.200*** 


0 179*** 


0.318*** 


0.290*** 




[0.014] 


[0.014] 


[0.007] 


[0.007] 


Other race 


-0.024 


-0.029* 


0.170*** 


0.147*** 




[0.017] 


[0.017] 


[0.010] 


[0.010] 


HS dropout 


0.054*** 


0.034*** 


0.306*** 


0.298*** 




[0.013] 


[0.013] 


[0.007] 


[0.007] 


HS graduate 


0.142*** 


0.147*** 


0.186*** 


0.189*** 




[0.010] 


[0.010] 


[0.005] 


[0.005] 


College fixed effects 


No 


Yes 


No 


Yes 


R-squared 


0.022 


0.033 


0.091 


0.096 


Observations 


137,449 


137,449 


343,040 


343,040 



Note. OLS estimation. Unstandardized coefficients reported. Standard errors in parentheses. 
*p < 0.1. **p < 0.05. ***p < 0.01. 
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Among full-time students, males, high school dropouts, and Black students were 
less likely to accumulate points. Both high school dropouts and high school graduates 
were more likely to gain points as part-time students. However, part-time and full-time 
students appear equally important to colleges’ ability to raise points. 

Notably, all student characteristics examined here were statistically significant 
influences on points accumulation. However, these characteristics only explain 2-10 
percent of the variation in points per student. 

Moreover, there was considerable heterogeneity of effects across each individual 
metric. No particular college had an easier way to accumulate points. Tables A. 7 and A. 8 
report estimates for earning any points in each of the six metrics by full-time and part- 
time student status for the 2011 academic year (including college fixed effects). 

Consistent with the principles of the SAI, basic skills points appear to operate as a 
way to serve disadvantaged groups and counterbalance the accumulation of points by 
other, more advantaged student groups. Among full-time students (Table A. 7), those on 
financial aid were likely to accumulate more total points but less likely to accumulate 
basic skills points. This result is reasonable because basic skills courses are not eligible 
for financial aid. Younger students were less likely to accumulate basic skills points, as 
were White students. There was a similar pattern for high school dropouts. Thus, 
unsurprisingly, the basic skills metric appears to be the most powerful in encouraging 
enrollment from traditionally underserved groups. 

Other groups were also weighted toward particular metrics. Male students were 
more likely to accumulate 15 -credit and 30-credit points as well as quantitative reasoning 
points; they were less likely to accumulate basic skills, college readiness, and degree 
completion points. Overall, it appears that five of the six metrics are detennined by 
similar characteristics; the exception is basic skills, which is influenced by other 
countervailing student characteristics. Thus, basic skills points accumulation 
counterbalances the influences on the other metrics. 
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5. Alignment of Alternative Measures 



One way to assess the SAI is to see how it implicitly orders the colleges in terms 
of performance. The SAI is explicitly intended not to rank colleges, and in general, 
ordinal ranking is often not the best way to understand differences between colleges 
(Jenkins & Cho, 2012). The SAI rewards colleges based on their improvement against 
their own baselines, not against the performances of other colleges; it is a funding system, 
not an ordering system. Still, performance systems should be robust (i.e., insensitive to 
variations in how they are calculated), and the SAI does allocate funds based on colleges’ 
relative performance. Some colleges receive more funds than others. Thus, the allocation 
system of the SAI will be more credible if it reflects a valid measure of relative 
perfonnance, i.e., if colleges that receive more funds using the SAI formula would also 
receive more funds based on an alternative, equally plausible formula. 

A prior analysis (Belfield, 2011) found that using alternative methods to identify 
relative performance yielded a different ordering for the colleges. 19 For example, under 
the SAI, the highest performing college was College 028. Using two alternative 
efficiency-related measures, College 028 ranked 20th or 28th, and using two 
effectiveness measures, it ranked 25th or 32nd. 20 The current analysis looks at rankings 
over time using the SAI allocation rule and at rankings across different allocation rules 
for the 2009 year (the one for which the most recent financial data were available). 

Under the SAI, the highest performing college is the one that increased its 
absolute total of points the most between the baseline year and 2009 and then each 
subsequent year. Table 10 shows the ranks of the colleges based on the SAI formula. 
There is some stability in the rankings; those colleges that ranked highly from baseline- 
2009 also ranked highly from baseline-201 1 (with a Pearson correlation coefficient of 

18 Crucially, colleges should not accumulate more points if the marginal revenue is below the marginal cost 
of claiming the points. For example, if a college has a quantitative reasoning class that is fully enrolled, it 
may be too costly to offer another section of that class if a few extra students turn up. The largest college 
need not be the most efficient (or even be balancing its budget) if the net cost of enrolling a student exceeds 
the marginal revenue. 

19 These methods included ranking colleges on the basis of total points change, baseline-2009; total points 
change, 2007-2008; percentage total points change, baseline-2009; total points per dollar; average points 
in 2007; residuals from an equation determining total points; and percentage change in average points, 
baseline-2009. 

20 Efficiency measures are based on the residuals from an equation determining total points and total points 
per dollar of expenditure. Effectiveness measures are based on total points per student and percentage 
change in average points. 



27 




0.64). Only one college had a top-ten ranking for all four years (College 002), and no 
single college had a bottom-ten ranking in all four years. Overall, there is no strong 
polarization over time into high-ranking and low-ranking colleges when the SAI is 
interpreted as a ranking system. Moreover, the funding implications are even less 
polarizing because the funding fonnula is not as strongly discriminating as a ranking 
system is. That is, a ranking system discriminates between every college. In 201 1, the 
SAI funding fonnula allocated awards to only 21 colleges; the remaining 13 colleges 
were treated equally (i.e., they received no SAI funding). 

There are multiple ways that colleges’ relative perfonnance within the 
Washington State system could be measured. Alongside the SAI fonnula, four ranking 
systems are examined here: three in -levels” (expenditure per point; points per student; 
and points per full-time equivalent student) and one in — feanges” (changes in points per 
FTE 2007-09). The first three levels measures are based on what colleges do, rather than 
what they have done compared to their past performance. These comparisons are 
presented to illustrate the full range of possible alternatives. Note that levels and changes 
cannot be simply evaluated against each other. 

Table 1 1 shows the ranks of the colleges using these alternative ranking systems 
for 2009. The correlations across the ranking systems are given in Table 12. The ranking 
system based on absolute change in points yields a different ordering than the other three 
methods. The other three methods yield ordering systems which are similar to one 
another, although this is in part because they are constructed using similar variables (e.g., 
student numbers and FTE numbers) and because they are based on levels. Thus, it 
appears that increasing points over time involves different practices than accumulating a 
great number of points each year. 

A more direct comparison with the current ranking system is ranking based on 
change in points per FTE 2007-09; see Table 11. This system yields different rankings 
than measuring absolute change in total points. Some of the colleges that are ranked very 
low based on absolute change in total points — such as College 032 and College 033 — are 
ranked very high when the ranks are based on changes in points per FTE. As shown in 
Table 12, the correlation between these two ranking systems is strongly negative (-0.56). 
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Table 10 

College Rankings Based on the SAI Formula 



College 




Total Points Change 




2007-09 


2009-10 


2010-11 


2007-11 


College 001 


24 


11 


33 


28 


College 002 


7 


5 


3 


3 


College 003 


16 


24 


16 


18 


College 004 


23 


29 


22 


30 


College 005 


28 


23 


19 


27 


College 006 


33 


27 


9 


31 


College 007 


4 


1 


28 


1 


College 008 


11 


9 


23 


12 


College 009 


6 


34 


6 


26 


College 010 


2 


15 


13 


5 


College Oil 


5 


3 


29 


6 


College 012 


32 


25 


24 


33 


College 013 


8 


13 


11 


8 


College 014 


3 


2 


32 


2 


College 015 


22 


21 


21 


22 


College 016 


14 


6 


25 


11 


College 017 


12 


7 


5 


7 


College 018 


15 


33 


12 


25 


College 019 


17 


26 


14 


21 


College 020 


31 


17 


4 


17 


College 021 


25 


19 


17 


20 


College 022 


18 


4 


26 


9 


College 023 


26 


28 


8 


24 


College 024 


9 


14 


34 


19 


College 025 


21 


22 


27 


29 


College 026 


27 


20 


2 


14 


College 027 


20 


32 


30 


34 


College 028 


1 


30 


31 


13 


College 029 


19 


16 


18 


16 


College 030 


10 


12 


1 


4 


College 031 


30 


18 


15 


23 


College 032 


34 


31 


10 


32 


College 033 


29 


10 


7 


15 


College 034 


13 


8 


20 


10 
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Table 11 

College Ranks Based on Alternative Measures 



College 






Basis of Rank 






Absolute 
Change in Total 
Points, 2007-09 


Expenditure Per 
Point, 2009 


Points Per 
Student, 2009 


Points Per FTE, 
2009 


Change in 
Points Per FTE, 
2007-09 


College 001 


24 


32 


34 


34 


18 


College 002 


7 


24 


29 


24 


11 


College 003 


16 


28 


32 


30 


32 


College 004 


23 


22 


1 


5 


33 


College 005 


28 


8 


4 


7 


6 


College 006 


33 


16 


17 


13 


1 


College 007 


4 


11 


8 


16 


14 


College 008 


11 


27 


33 


33 


21 


College 009 


6 


5 


3 


12 


22 


College 010 


2 


15 


5 


6 


25 


College Oil 


5 


6 


13 


4 


30 


College 012 


32 


25 


15 


13 


12 


College 013 


8 


14 


10 


9 


29 


College 014 


3 


10 


9 


19 


31 


College 015 


22 


4 


11 


10 


20 


College 016 


14 


26 


18 


25 


15 


College 017 


12 


13 


14 


22 


13 


College 018 


15 


12 


23 


2 


34 


College 019 


17 


7 


2 


7 


22 


College 020 


31 


2 


27 


1 


19 


College 021 


25 


3 


26 


17 


24 


College 022 


21 


21 


22 


27 


17 


College 023 


26 


18 


30 


23 


9 


College 024 


9 


20 


28 


26 


28 


College 025 


20 


29 


16 


31 


16 


College 026 


27 


30 


25 


28 


2 


College 027 


19 


34 


21 


32 


10 


College 028 


1 


19 


19 


20 


27 


College 029 


18 


23 


31 


18 


5 


College 030 


10 


17 


6 


21 


7 


College 031 


30 


33 


24 


29 


8 


College 032 


34 


31 


12 


15 


3 


College 033 


29 


1 


20 


3 


3 


College 034 


13 


9 


7 


11 


26 



Note. Higher ranks are associated with better performance. Rank 1 for expenditure per point is the college with the 
lowest expenditure per point. 
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Table 12 

Correlations Between Ranking Systems 



Basis of Rank 


Absolute 
Change in Total 
Points, 2007-09 


Expenditure 
Per Point 


Points Per 
Student, 2009 


Points Per 
FTE, 2009 


Expenditure per point 


0.17 








Points per student, 2009 


0.23 


0.42 






Points per FTE, 2009 


-0.03 


0.76 


0.56 




Change in points per FTE, 2007-09 


-0.56 


-0.26 


-0.14 


-0.21 



6. Momentum Analysis 

This section focuses on momentum — getting students further along a college 
pathway. - In this analysis, it is critical to consider where students started in order to 
predict how many points they might earn. I relate the idea of momentum to points by 
depicting a pathway where students are defined in terms of their highest level of points. 
Students enter the path with no points. The first points they accumulate along the path are 
either basic skills points or college readiness points. They move on to accumulate points 
for reaching 15 and then 30 credits, or for passing a gatekeeper quantitative reasoning 
course. Finally, students at the end of the pathway receive credential or qualification 
points. Momentum refers to students’ progress along this pathway of points. For 
simplicity, students are referred to in terms of their points; for example, a college -ready 
student is one who has passed pre-college remedial or developmental courses (rather than 
one who did not need to take a remedial course). It is important to account for dropouts; 
students who drop out obviously do not have momentum. Below, dropouts are included 
in the counts of those not progressing along a pathway. 

The analysis above suggests that, in the aggregate, student momentum may not 
have changed much over the years. As shown in Figure 2, the proportion of points that 
are accumulated toward the beginning of the pathway — basic skills or college readiness 
points — remained stable. If colleges were increasing momentum, a greater proportion of 
their points would have been gained from the metrics that take place later in students’ 



21 I also investigated the possibility that students are accumulating more points within a given year. 
However, no strong differences were evident. 
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academic pathways (e.g., 30 credits and degree completion). Interesting patterns result 
from examining the data at a more disaggregated level and from looking at a cohort of 
students over time. 

In order to look at momentum for individual students tracked longitudinally, I 
analyze outcomes for a cohort of students in Washington community colleges in a given 
year. These students are categorized as basic skills, transfer, or workforce students, 
following the definitions used by the Washington State Board for Community and 
Technical Colleges. Basic skills (B) students can earn any category of points. Transfer 
(T) students and workforce (W) students typically do not earn basic skills points but can 
earn college readiness points if they enroll in and complete college remedial courses. 
Approximately 15 percent are B students, 34 percent are T students, and 51 percent are 
W students. On average, each student accumulates 0.7 points." To identify momentum, 
student progress is tracked from a start year into the following year to see how many 
points were accumulated. (Results are only reported for one year to the next because few 
students persist for more than two years.) As noted above, the pathway of progression in 
points is: none, basic skills, college readiness, 15 credits, 30 credits, quantitative 
reasoning, and completion. The tracks are based on the students’ status in the start year; 
it is possible to see, for example, how many of the college-ready students in 2007 
completed 15 credits in 2008. Rates of progress can then be compared across years to see 
if the sector is generating more momentum along a pathway for a given set of college- 
ready students. 

Full tabulations of the progressions are given in Table A. 9 for each of the student 
types across three start years: 2007, 2009, and 2010. Figures 5-7 illustrate patterns for the 
2009 start year. 



In addition, there are students in the intent code category -ether.” This category covers approximately 20 
percent of all students, but since they accumulate very few points, they are not part of this analysis. While 
the average B, T, or W student accumulates 0.7 points per year, students in the other category accumulate 
0.1 points per year. 

23 A student who gained basic skills and college ready points in 2010 is designated as college ready. 
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Figure 5 

Distribution of Points in 2010 for Cohort of Students 
with Zero Points in 2009 




■ Certification 

■ Quants 

■ Credit 30 

■ Credit 15 

■ College ready 

■ Basic skills 

■ Zero points 



Figure 6 

Distribution of Points in 2010 for Cohort of Students 
with College Readiness Points in 2009 




■ Certification 

■ Quants 

■ Credit 30 

■ Credit 15 

■ College ready 



33 



Figure 7 

Distribution of Points in 2010 for Cohort of Students 
with 15 Credit Points in 2009 




Results for the more than 50 percent of students who did not have any points in a 
given year are displayed in column 1 of Table A. 9. Figure 5 shows the points 
accumulations in 2010 for the 2009 cohort of students who had zero points. Of those with 
zero points in 2009, over 80 percent did not have any points in 2010. Most of these 
students who did not earn points in either year were not enrolled in 2010; this is one way 
that students can lack momentum. For the remaining students, pathways varied by 
category. For B students with no points in 2009, 9 percent had basic skills points, with 
tiny percentages progressing further. For T students, 18 percent subsequently 
accumulated points, which were spread across all metrics except basic skills; 7 percent of 
these students completed some type of credential by 2010. For W students, 13 percent 
accumulated points in 2010; the points were gained from all six metrics, with 5 percent 
gaining a degree or certificate by 2010. Therefore, if a student accumulates no points in 
the first year — as is the case for more than 50 percent of all students — it is very unlikely 
that the student will accumulate any points, at least over a two-year period. 

In each year, 9 percent of all students had basic skills points. In the following 
year, very few of these students accumulated any additional points; in 2010, less than 3 
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percent became college ready, and an even smaller fraction reached 15 credits or beyond. 
The progression rates were similar across B, T, and W students (See Table A. 9). 
Therefore, almost all the students who get basic skills points in a given year subsequently 
accumulate no more points. 

Students with college readiness points exhibited more progression each year; 
these students did have some momentum. Figure 6 shows the paths of those students who 
were college ready in 2009. For B students, over one quarter gained additional points in 
2010: 10 percent reached 15 credits; 8 percent reached 30 credits; and 2 percent 
completed their program. For T students, almost one quarter gained additional points in 
2010, with respective progression rates of 7 percent, 6 percent, and 7 percent. W students 
progressed the furthest: one third gained points in 2010, with 8 percent reaching 15 
credits, 10 percent reaching 30 credits, and 7 percent gaining a credential. 

Students with 15 credits also progressed. Generally, momentum increases as 
students progress along a pathway. Figure 7 shows the paths of those students who had 
15 credits in 2009. For B students, one quarter of those with 15 credits in 2009 
accumulated 30 credits in 2010 (with 4 percent completing a credential). Similar 
progression was made by T students; one quarter reached 30 credits, and 2 percent 
completed a credential. The progression rate for W students was slightly higher: 28 
percent obtained 30 credits, and 5 percent obtained a credential. Unsurprisingly, attrition 
rates were much lower for students who already had points. Since these students 
represent less than 40 percent of the student body, it is difficult for an improvement in 
their momentum (e.g., more 15-credit students getting to 30-credits) to significantly 
increase the total points a college can obtain or the average number of points per student. 

Looking across the three start years — 2007, 2009, and 2010 — some differences 
are apparent, but there are no strong trends. The inertia of students with zero points and 
basic skills points appears constant. However, there are two exceptions. First, more 
transfer students with basic skills points in 2010 made progress — 14 percent by 201 1, 
compared with 7 percent from 2007 to 2008. Considered in terms of absolute numbers of 
students, though, these figures are very small. Second, students of all types with 15 credit 
points progressed at higher rates in 2010 than equivalent students did in 2007. These 
exceptions indicate that, modestly, colleges are increasing momentum over time. 
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The findings for transfer and workforce students are not sensitive to their pre- 
college pathways. Those students who were in pre-college courses progressed in a similar 
fashion to those who entered directly into college-level classes. Table A. 10 replicates the 
pathways of only those T and W students who attempted at least one pre-college English 
or math course. The rates of progression are similar to those for all T and W students. 



7. Conclusion 

This paper has examined, from a variety of empirical perspectives, total points 
and changes in points across the state’s colleges during the initial phase of the SAI 
between 2007 and 2011. The findings from this analysis accord closely with the 
conclusions from prior analyses (Belfield, 2009, 2011). 

Colleges increased their total points each year through 2010, although this trend 
stalled in 201 1. From baseline to 201 1, the average college increased its total points by 
3 1 percent. It also increased its points per student at nearly the same rate (29 percent). At 
the same time, colleges increased enrollment; the number of FTEs grew by 41 percent, 
indicating that students were enrolling more intensively. The net result is that colleges’ 
points per FTE fell by 4 percent since baseline. In the first three years of the SAI, 
colleges were becoming more effective — getting more points from each student — but this 
trend halted in 20 1 1 . 

As with prior analyses, there is no strong evidence that the method of allocating 
awards based on change in total points significantly favors particular colleges. No college 
had fewer points in 201 1 than at baseline. Some colleges had bigger changes in points, 
but this did not result in drastic changes in their rankings based on total points. Because 
there was some volatility in each year, colleges neither converged nor diverged in terms 
of gains in points. In terms of points per student, colleges converged over time. This may 
indicate the presence of a plateau beyond which colleges may find it difficult to continue 
increasing points. Over time, it will be possible to investigate this. 

Enrollment size and total points were positively correlated for every year of data. 
Size influences total points, and while there are indications that size is becoming slightly 
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more influential over time, the effect is not dramatic. Although larger colleges had more 
points, they did not have more points per student. 

The six metrics appear to be stable in their importance. In the first few years, 
colleges trended toward accumulating more of their points from gains in basic skills and 
college readiness. After the first few years, though, there was not much overall change in 
the proportions of students accumulating points across each metric. Insofar as the SAI is 
intended to increase incentives for serving disadvantaged groups, the basic skills point 
demonstrates the importance of this mission area to following the principle. However, the 
above analysis of momentum suggests that this point accumulation in and of itself has 
little or no connection to the goal of raising postsecondary achievement. 

The characteristics of the student body at the colleges did not strongly influence 
total points and the change in points. Indeed, none of the characteristics of the student 
body — such as age, full-time enrollment status, race, or prior education — were associated 
with total points or with change in points. Although there were some significant 
influences on points across each of the six metrics, these differences do not result in large 
effects when the data are viewed in aggregate. In addition, the analysis of points per 
student suggests that the points system does not favor particular types of student. The 
influence of other characteristics depends on whether the student is classified as full-time 
or part-time. All of these conclusions echo those found in Belfield (2011). 

Similarly, no simple model predicts points accumulation for any given student; it 
is not obvious which students yield the most points. Thus, colleges are unlikely to be able 
to target particular groups for enrollment. This was a key principle of the SAI. By default, 
this finding implies that the best way to accumulate points is to increase momentum — i.e., 
ensure that students who are college ready persist to 15 credits, 30 credits, and then to 
graduation. This, too, corresponds to the intent of the SAI. 

The findings on the SAI as a ranking system are less clear-cut. The SAI is not 
intended as a ranking system, but colleges that rank higher do receive more funding. The 
use of alternative methods for evaluating the perfonnance of each college — based either 
on levels of points or changes in points from one year to the next — would lead to 
different orderings. In turn, this would lead to somewhat different funding allocations. 
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In spite of the observed trend toward increases in points, colleges may have 
difficulty in substantially improving their point totals by generating extra momentum. 
This difficulty reflects the structure of the points-based award system rather than the 
institutional capacity of the colleges. Approximately 50 percent of students did not 
accumulate any points in a given year, and these students were unlikely to accumulate 
points in any subsequent year (in large part because they tended not to enroll). It may be 
difficult to get these students to progress far in college. Moreover, among students who 
accumulated basic skills points in any given year, few accumulated points in the 
following year. Only a small proportion of each student cohort progressed along a 
pathway of points — moving from 15 to 30 credits, for example. Consequently, colleges 
that retain more students may actually find it harder to accumulate points for these 
students beyond their first year. Colleges may be more likely to improve student 
momentum if they focus on the gains that can be made among new students during their 
first year in college. 
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Appendix: Tables 



Table A.l 

Points Per College, Year 2009 



Student 

Characteristics 


Total Points 


Basic Skills 


College 

Readiness 


15 Credits 


30 Credits 


Quantitative 

Reasoning 


Degree 

Completion 


Financial aid 


-862 


-2,019 


3,475 


-2,829 


-1,351 


52 


1,810 




(9,626) 


(7,572) 


(4,875) 


( 1 , 888 ) 


(1,379) 


(1,840) 


(1,180) 


SES bottom quintile -4,243 


-2,159 


-2,235 


286 


94 


-590 


361 




(4,074) 


(3,205) 


(2,063) 


(799) 


(584) 


(779) 


(500) 


Age < 20 


1,260 


-936 


1,117 


115 


555 


1,458 


-1,049 




(6,094) 


(4,793) 


(3,086) 


(1,195) 


(873) 


(1,165) 


(747) 


Age > 65 


6,672 


15,178 


6,430 


-6,723* 


-3,985 


445 


-4,673* 




(19,231) 


(15,127) 


(9,740) 


(3,771) 


(2,755) 


(3,676) 


(2,358) 


Disability 


10,993 


27,484 


-7,232 


-3,931 


-2,871 


-2,033 


-424 




(24,526) 


(19,292) 


(12,421) 


(4,809) 


(3,514) 


(4,688) 


(3,007) 


Full-time 


18,308** 


6,158 


3,916 


3,209** 


2,698** 


1,343 


984 




(7,166) 


(5,637) 


(3,629) 


(1,405) 


(1,027) 


(1,370) 


(879) 


Male 


-226 


7,316 


-5,168* 


-697 


-448 


-2,108* 


879 




(5,658) 


(4,451) 


( 2 , 866 ) 


( 1 , 110 ) 


(811) 


(1,082) 


(694) 


Asian 


13,097 


12,307 


-3,035 


-2,265 


-586 


4,680* 


1,997 




(12,828) 


(10,091) 


(6,497) 


(2,515) 


(1,838) 


(2,452) 


(1,573) 


Black 


-7,153 


3,640 


818 


-1,572 


-2,016 


-4,247 


-3,776** 




(14,542) 


(11,439) 


(7,365) 


(2,851) 


(2,083) 


(2,780) 


(1,783) 


Hispanic 


2,919 


9,505** 


-1,594 


-2,266** 


-1,598** 


-92 


-1,036* 




(4,323) 


(3,401) 


(2,189) 


(848) 


(619) 


(826) 


(530) 


Other race 


10,027 


11,147 


-1,411 


166 


382 


606 


-863 




(13,600) 


(10,698) 


( 6 , 888 ) 


(2,667) 


(1,948) 


(2,600) 


( 1 , 668 ) 


HS dropout 


9,816 


7,006 


3,024 


387 


-116 


-1,228 


744 




(7,037) 


(5,535) 


(3,564) 


(1,380) 


(1,008) 


(1,345) 


(863) 


HS graduate 


4,244 


-10,777* 


5,806 


3,739** 


2,690** 


2,863* 


-77 




(7,732) 


(6,082) 


(3,916) 


(1,516) 


(1,108) 


(1,478) 


(948) 


Enrollment 


672*** 


168*** 


247 *** 


138*** 


102 *** 


74 *** 


42 *** 




(51) 


(40) 


(26) 


( 10 ) 


(7) 


( 10 ) 


( 6 ) 


N 


34 


34 


34 


34 


34 


34 


34 


R-squared 


0.939 


0.788 


0.785 


0.943 


0.947 


0.889 


0.817 



Note. Enrollment is in thousands. 
*p < 0.1. **p < 0.05. ***p < 0.01. 
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Table A.2 

Points Per College, Year 2010 



Student 

Characteristics 


Total Points 


Basic Skills 


College 

Readiness 


15 Credits 


30 Credits 


Quantitative 

Reasoning 


Degree 

Completion 


Financial aid 


3,446 


2,079 


3,251 


-2,160 


-715 


-257 


1,248 




(10,475) 


(9,788) 


(6,431) 


(1,979) 


(1,665) 


(1,749) 


(1,503) 


SES bottom quintile -11,249** 


-5,726 


-4,646* 


-164 


-121 


-105 


-486 




(4,198) 


(3,923) 


(2,577) 


(793) 


(667) 


(701) 


(603) 


Age < 20 


-4,188 


-6,458 


-1,530 


1,650 


1,623 


1,426 


-899 




(6,061) 


(5,664) 


(3,721) 


(1,145) 


(963) 


( 1 , 012 ) 


(870) 


Age > 65 


-234 


21,984 


-2,985 


-9,551** 


-6,273** 


-416 


-2,993 




(17,752) 


(16,589) 


(10,898) 


(3,355) 


(2,821) 


(2,965) 


(2,548) 


Disability 


17,160 


24,887 


-5,554 


-1,898 


229 


-391 


-113 




(24,692) 


(23,073) 


(15,158) 


(4,666) 


(3,924) 


(4,124) 


(3,544) 


Full-time 


15,057* 


6,402 


3,893 


1,679 


1,588 


-23 


1,519 




(7,673) 


(7,170) 


(4,710) 


(1,450) 


(1,219) 


(1,281) 


( 1 , 101 ) 


Male 


1,158 


10,448 


-7,747* 


-1,092 


-56 


-1,360 


965 




( 6 , 868 ) 


(6,417) 


(4,216) 


(1,298) 


(1,091) 


(1,147) 


(986) 


Asian 


-11,600 


2,753 


-12,751 


-4,558* 


-2,898 


5,101** 


753 




(13,917) 


(13,005) 


(8,544) 


(2,630) 


( 2 , 212 ) 


(2,324) 


(1,998) 


Black 


12,174 


16,180 


3,206 


-605 


641 


-5,110** 


-2,137 




(14,540) 


(13,587) 


(8,926) 


(2,747) 


(2,311) 


(2,428) 


(2,087) 


Hispanic 


7,085 


13,562*** 


-2,095 


-2,055** 


-1,614** 


-457 


-257 




(4,569) 


(4,270) 


(2,805) 


(863) 


(726) 


(763) 


(656) 


Other race 


55,591*** 


46,743** 


8,844 


-313 


-1,963 


2,468 


-188 




(18,696) 


(17,471) 


(11,478) 


(3,533) 


(2,971) 


(3,122) 


(2,684) 


HS dropout 


22,390*** 


15,772** 


7,737* 


-197 


128 


- 2,021 


971 




(7,064) 


(6,601) 


(4,337) 


(1,335) 


(1,123) 


(1,180) 


(1,014) 


HS graduate 


-6,317 


-14,290* 


3,701 


1,780 


784 


2,273* 


-565 




(7,637) 


(7,136) 


(4,688) 


(1,443) 


(1,214) 


(1,275) 


(1,096) 


Enrollment 


722*** 




154*** 


245 *** 


107*** 


63*** 


44* * * 




(47) 


(44) 


(29) 


(9) 


(7) 


(8) 


(7) 


N 


34 


34 


34 


34 


34 


34 


34 


R-squared 


0.949 


0.795 


0.762 


0.956 


0.941 


0.905 


0.763 



Note. Enrollment is in thousands. 
*p < 0.1. **p < 0.05. ***p < 0.01. 
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Table A.3 

Points Per College, Year 2011 



Student 

Characteristics 


Total Points 


Basic Skills 


College 

Readiness 


15 Credits 


30 Credits 


Quantitative 

Reasoning 


Degree 

Completion 


Financial aid 


10,938 


2,310 


7,404 


-1,055 


357 


239 


1,685 




(8,379) 


(6,340) 


(5,516) 


(1,297) 


(915) 


(1,239) 


(1,124) 


Age < 20 


5,186 


-4,717 


3,355 


2,308** 


1,844** 


3,195*** 


-798 




(7,059) 


(5,341) 


(4,647) 


(1,093) 


(771) 


(1,044) 


(947) 


Age > 65 


-6,755 


16,715 


-7,449 


-6,654** 


-2,891 


-235 


-6,236** 




(18,270) 


(13,824) 


(12,027) 


(2,828) 


(1,995) 


(2,702) 


(2,451) 


Disability 


4,553 


21,249 


-9,865 


-3,301 


-3,265 


508 


-777 




(23,810) 


(18,016) 


(15,674) 


(3,686) 


(2,600) 


(3,521) 


(3,195) 


Full-time 


-718 


2,177 


-2,381 


-340 


-444 


-1,047 


1,316 




(6,830) 


(5,167) 


(4,496) 


(1,057) 


(746) 


(1,010) 


(916) 


Male 


5,330 


10,347* 


-5,581 


-179 


115 


-1,712 


2,341** 




(7,158) 


(5,416) 


(4,712) 


(1,108) 


(782) 


(1,059) 


(960) 


Asian 


10,664 


11,391 


-2,859 


-2,315 


-447 


3,876** 


1,019 




(11,739) 


(8,882) 


(7,728) 


(1,817) 


(1,282) 


(1,736) 


(1,575) 


Black 


-1,167 


9,164 


518 


-2,880 


-2,455* 


-3,042 


-2,472 




(12,389) 


(9,374) 


(8,155) 


(1,918) 


(1,353) 


(1,832) 


(1,662) 


Hispanic 


4,564 


12,606*** 


-4,240 


-1,447* 


-1,102** 


-459 


-793 




(4,744) 


(3,590) 


(3,123) 


(735) 


(518) 


(702) 


(637) 


Other race 


33,091 


29,239* 


-920 


2,652 


1,591 


1,580 


-1,055 




(20,219) 


(15,298) 


(13,310) 


(3,130) 


(2,208) 


(2,990) 


(2,713) 


HS dropout 


14,707* 


11,839** 


6,582 


-1,108 


-1,389* 


-2,500** 


1,282 




(7,285) 


(5,512) 


(4,795) 


(1,128) 


(796) 


(1,077) 


(977) 


HS graduate 


-8,808 


-14,034** 


1,129 


2,147* 


1,836** 


1,618 


-1,505 




(7,648) 


(5,786) 


(5,034) 


(1,184) 


(835) 


(1,131) 


(1,026) 


Enrollment 


718*** 


17g*** 


164*** 


^41*** 


no*** 


gg*** 


55*** 




(49) 


(37) 


(32) 


(8) 


(5) 


(7) 


(7) 


N 


34 


34 


34 


34 


34 


34 


34 


R-squared 


0.943 


0.804 


0.715 


0.966 


0.972 


0.926 


0.832 



Note. Enrollment is in thousands. *p < 0.1. **p < 0.05. ***p < 0.01. 
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Table A.4 

Change in Total Points Per College 



Student Characteristics 


Change in 
Points 07-09 


Change in 
Points 09-10 


Change in 
Points 10-11 


Change in 
Points 07-11 


Financial aid 


5,621 


3,766 


-2,564 


256 




(7,267) 


(7,250) 


(5,405) 


(13,348) 


SES bottom quintile 


-4,872** 


540 


45 


-3,265 




(2,296) 


(3,068) 


(2,166) 


(4,218) 


Age < 20 


1,815 


-3,306 


909 


-1,153 




(3,966) 


(4,589) 


(3,128) 


(7,285) 


Age > 65 


-1,865 


-11,623 


7,601 


1,355 




(10,688) 


(14,483) 


(9,161) 


(19,632) 


Disability 


21,188* 


-25,234 


130 


-1,028 




(11,984) 


(18,470) 


(12,742) 


(22,012) 


Full-time 


1,776 


-2,092 


-3,936 


336 




(3,882) 


(5,397) 


(3,960) 


(7,131) 


Male 


699 


-2,782 


-3,986 


-8,855 




(3,164) 


(4,261) 


(3,544) 


(5,811) 


Asian 


-8,413 


-1,039 


3,375 


-4,633 




(7,292) 


(9,661) 


(7,182) 


(13,394) 


Black 


10,712 


-3 


-5,136 


8,588 




(8,250) 


(10,952) 


(7,503) 


(15,153) 


Hispanic 


-1,826 


-5,439 


1,307 


-3,956 




(2,894) 


(3,256) 


(2,358) 


(5,316) 


Other race 


-2,295 


19,616* 


-3,610 


6,664 




(9,310) 


(10,242) 


(9,648) 


(17,101) 


HS dropout 


11,177* 


9,699* 


-4,987 


13,345 




(5,549) 


(5,300) 


(3,645) 


(10,192) 


HS graduate 


-5,931 


-1,371 


6,592 


1,281 




(5,545) 


(5,823) 


(3,941) 


(10,185) 


Base enrollment 


107 *** 


82** 


-50** 


165*** 




(27) 


(38) 


(23) 


(39) 


N 


34 


34 


34 


34 


R-squared 


0.656 


0.536 


0.447 


0.689 



Note. Enrollment is in thousands. 
*p < 0.1. **p < 0.05. ***p < 0.01. 
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Table A.5 

Points Per Full-Time Equivalent 



College 


Baseline 
AY 2007 


AY 2009 


AY 2010 


AY 2011 


% Change 
07-09 


% Change 
09-10 


% Change 
10-11 


% Change 
07-11 


College 001 


1.21 


1.35 


1.53 


1.47 


12 


13 


-4 


22 


College 002 


2.21 


2.25 


1.75 


1.82 


2 


-22 


4 


-18 


College 003 


1.54 


2.01 


1.94 


1.91 


31 


-3 


-1 


24 


College 004 


2.61 


3.12 


2.61 


2.44 


20 


-16 


-6 


-6 


College 005 


2.96 


2.91 


2.43 


2.27 


-2 


-17 


-6 


-23 


College 006 


2.72 


2.52 


2.24 


2.22 


-7 


-11 


-1 


-18 


College 007 


2.36 


2.46 


2.19 


2.07 


4 


-11 


-5 


-12 


College 008 


1.38 


1.56 


1.58 


1.54 


13 


2 


-3 


12 


College 009 


2.35 


2.54 


2.06 


2.14 


8 


-19 


4 


-9 


College 010 


2.79 


3.07 


2.52 


2.44 


10 


-18 


-3 


-13 


College Oil 


2.83 


3.18 


2.54 


2.33 


12 


-20 


-9 


-18 


College 012 


2.46 


2.52 


2.29 


2.12 


2 


-9 


-7 


-14 


College 013 


2.37 


2.69 


2.15 


2.08 


14 


-20 


-4 


-12 


College 014 


1.98 


2.38 


2.34 


2.28 


20 


-2 


-3 


15 


College 015 


2.46 


2.63 


2.24 


2.17 


7 


-15 


-3 


-12 


College 016 


2.11 


2.22 


2.06 


2.00 


5 


-7 


-3 


-5 


College 017 


2.22 


2.32 


2.18 


2.14 


5 


-6 


-2 


-4 


College 018 


2.71 


3.40 


2.23 


2.33 


25 


-34 


5 


-14 


College 019 


2.72 


2.91 


1.16 


1.13 


7 


-60 


-2 


-58 


College 020 


3.33 


3.47 


4.45 


4.63 


4 


28 


4 


39 


College 021 


2.19 


2.42 


2.40 


2.39 


11 


-1 


-1 


9 


College 022 


2.08 


2.20 


2.26 


2.23 


6 


3 


-1 


7 


College 023 


2.24 


2.27 


2.05 


2.12 


1 


-10 


4 


-5 


College 024 


1.89 


2.21 


2.14 


1.93 


17 


-3 


-10 


2 


College 025 


1.77 


1.89 


2.05 


1.90 


7 


8 


-7 


7 


College 026 


2.22 


2.08 


2.04 


2.28 


-6 


-2 


11 


3 


College 027 


1.67 


1.70 


2.23 


2.03 


2 


31 


-9 


22 


College 028 


2.04 


2.36 


2.58 


2.59 


16 


9 


0 


27 


College 029 


2.49 


2.41 


1.18 


1.18 


-3 


-51 


0 


-53 


College 030 


2.37 


2.33 


2.21 


2.27 


-2 


-5 


3 


-4 


College 031 


2.05 


2.07 


1.98 


2.01 


1 


-4 


2 


-2 


College 032 


2.63 


2.50 


2.00 


1.89 


-5 


-20 


-6 


-28 


College 033 


3.48 


3.35 


2.61 


2.50 


-4 


-22 


-4 


-28 


College 034 


2.28 


2.58 


2.58 


2.65 


13 


0 


3 


16 


Average 


2.32 


2.47 


2.20 


2.16 


7 


-9 


-2 


-4 



Note. Points per student rounded to two decimal places. 
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Table A.6 

Any Non-Zero Points Per Individual Student, Academic Year 2011 





Full-Time Students 


Part-Time Students 


Student Characteristics 


(1) 


(2) 


(3) 


(4) 


Financial aid 


0.207*** 

[0.008] 


0.200*** 

[0.008] 


0.762*** 

[0.008] 


0.769*** 

[0.008] 


Age < 20 


0 729*** 
[0.044] 


0.758*** 

[0.045] 


1.114*** 

[0.011] 


1.094*** 

[0.011] 


Age 20-24 


0.321*** 

[0.044] 


0.351*** 

[0.045] 


0.857*** 

[0.010] 


0.841*** 

[0.010] 


Age 25-29 


0.348*** 

[0.044] 


0.379*** 

[0.046] 


0.775*** 

[0.011] 


0.758*** 

[0.011] 


Age 30-34 


0.358*** 

[0.045] 


0.389*** 

[0.046] 


0.692*** 

[0.011] 


0.675*** 

[0.011] 


Age 34-39 


0.359*** 

[0.045] 


0.389*** 

[0.047] 


0.657*** 

[0.012] 


0.642*** 

[0.012] 


Age 40-44 


0.391*** 

[0.046] 


0.416*** 

[0.047] 


0.622*** 

[0.013] 


0.611*** 

[0.013] 


Age 45-49 


0.398*** 

[0.047] 


0.423*** 

[0.048] 


0.579*** 

[0.013] 


0.575*** 

[0.013] 


Age 50-54 


0.467*** 

[0.048] 


0.486*** 

[0.049] 


0.403*** 

[0.014] 


0.405*** 

[0.014] 


Age 55-59 


0.401*** 

[0.051] 


0.423*** 

[0.052] 


0.217*** 

[0.016] 


0.224*** 

[0.017] 


Age 60-64 


0.345*** 

[0.061] 


0.364*** 

[0.062] 


-0.075*** 

[0.021] 


-0.052** 

[0.021] 


Disability 


-0.127*** 

[0.016] 


-0.125*** 

[0.016] 


0.144*** 

[0.013] 


0.147*** 

[0.013] 


Male 


-0.058*** 

[0.007] 


-0.056*** 

[0.007] 


0.036*** 

[0.005] 


0.034*** 

[0.005] 


Asian 


-0.001 

[0.013] 


0.023* 

[0.013] 


0.329*** 

[0.009] 


0.303*** 

[0.009] 


Black 


-0.178*** 

[0.014] 


-0.161*** 

[0.014] 


0.174*** 

[0.011] 


0.153*** 

[0.011] 


Hispanic 


-0.049*** 

[0.012] 


-0.066*** 

[0.012] 


0.149*** 

[0.008] 


0.131*** 

[0.008] 


Other race 


-0.102*** 

[0.014] 


-0.103*** 

[0.014] 


0.130*** 

[0.011] 


0.110*** 

[0.011] 


HS dropout 


-0.130*** 

[0.010] 


-0.132*** 

[0.011] 


0.204*** 

[0.007] 


0.195*** 

[0.008] 


HS graduate 


0.034*** 

[0.008] 


0.036*** 

[0.008] 


0.247*** 

[0.006] 


0.240*** 

[0.006] 


College fixed effects 


No 


Yes 


No 


Yes 


Observations 


137,449 


137,449 


343,040 


343,040 



Note. Probit estimation. Unstandardized coefficients reported. Standard errors in parentheses. 
*p < 0.1. **p < 0.05. ***p < 0.01. 
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Table A.7 

Any Non-Zero Points by Metric Per Individual Student: 
Full-Time Students, Academic Year 2011 



Student 

Characteristics 


Basic Skills 


College 

Readiness 


15 Credits 


30 Credits 


Quantitative 

Reasoning 


Degree 

Completion 


Financial aid 


-0.805*** 


0.534*** 


0.136*** 


0 191*** 


0.162*** 


0 . 122 *** 




[0.015] 


[0.008] 


[0.008] 


[0.008] 


[0.009] 


[ 0 . 010 ] 


Age < 20 


-0.721*** 


0.960*** 


0.583*** 


0.613*** 


1.268*** 


0.674*** 




[0.066] 


[0.079] 


[0.049] 


[0.052] 


[0.103] 


[0.087] 


Age 20-24 


-0 291*** 


0.569*** 


-0.119** 


0.074 


0.825*** 


0 979 *** 




[0.065] 


[0.079] 


[0.049] 


[0.052] 


[0.103] 


[0.087] 


Age 25-29 


-0.055 


0.569*** 


-0.113** 


0.073 


0.768*** 


0.967*** 




[0.066] 


[0.080] 


[0.049] 


[0.053] 


[0.103] 


[0.087] 


Age 30-34 


0.100 


0.544*** 


-0.166*** 


0.027 


0.747*** 


0 992*** 




[0.066] 


[0.080] 


[0.050] 


[0.053] 


[0.103] 


[0.088] 


Age 34-39 


0.141** 


0.524*** 


-0.170*** 


0.034 


0.684*** 


1.004*** 




[0.067] 


[0.080] 


[0.051] 


[0.054] 


[0.104] 


[0.088] 


Age 40-44 


0.214*** 


0.537*** 


-0.178*** 


0.033 


0.673*** 


0 977 *** 




[0.068] 


[0.081] 


[0.051] 


[0.055] 


[0.104] 


[0.089] 


Age 45-49 


0.216*** 


0.508*** 


-0.113** 


0.080 


0.690*** 


0.996*** 




[0.069] 


[0.081] 


[0.052] 


[0.055] 


[0.105] 


[0.089] 


Age 50-54 


0.233*** 


0.499*** 


-0.041 


0.089 


0.652*** 


1.034*** 




[0.070] 


[0.083] 


[0.053] 


[0.057] 


[0.106] 


[0.090] 


Age 55-59 


0.122 


0.424*** 


-0.073 


0.099* 


0.571*** 


1 029*** 




[0.074] 


[0.086] 


[0.056] 


[0.060] 


[0.109] 


[0.092] 


Age 60-64 


0.185** 


0.318*** 


-0.064 


0.039 


0.465*** 


0.925*** 




[0.085] 


[0.098] 


[0.067] 


[0.071] 


[ 0 . 121 ] 


[ 0 . 101 ] 


Disability 


0.007 


0.154*** 


-0.171*** 


-0.154*** 


-0.107*** 


-0.105*** 




[0.027] 


[0.017] 


[0.018] 


[0.018] 


[ 0 . 020 ] 


[ 0 . 021 ] 


Male 


-0.165*** 


-0.029*** 


0.061*** 


0.047*** 


0.070*** 


-0.104*** 




[ 0 . 012 ] 


[0.008] 


[0.007] 


[0.008] 


[0.008] 


[0.009] 


Asian 


0.654*** 


-0.065*** 


-0.163*** 


-0.168*** 


-0.009 


- 0 . 110 *** 




[0.019] 


[0.016] 


[0.014] 


[0.015] 


[0.016] 


[0.017] 


Black 


0.441*** 


0.000 


-0.160*** 


-0.240*** 


-0.345*** 


-0.367*** 




[0.023] 


[0.016] 


[0.016] 


[0.016] 


[ 0 . 020 ] 


[ 0 . 020 ] 


Hispanic 


0.643*** 


-0.009 


-0.169*** 


-0.252*** 


-0.249*** 


-0.332*** 




[0.017] 


[0.013] 


[0.013] 


[0.013] 


[0.015] 


[0.017] 


Other race 


0.273*** 


-0.013 


-0.088*** 


-0.135*** 


-0.127*** 


-0.251*** 




[0.023] 


[0.015] 


[0.015] 


[0.015] 


[0.017] 


[0.019] 


HS dropout 


0.642*** 


-0.209*** 


-0.319*** 


-0.169*** 


-0.138*** 


-0.140*** 




[0.016] 


[0.013] 


[ 0 . 012 ] 


[ 0 . 012 ] 


[0.013] 


[0.014] 


HS graduate 


0.044*** 


0.341*** 


- 0.010 


-0.062*** 


-0.098*** 


-0.186*** 




[0.015] 


[0.009] 


[0.008] 


[0.009] 


[ 0 . 010 ] 


[ 0 . 010 ] 


Observations 


137,449 


137,449 


137,449 


137,449 


137,449 


137,449 



Note. Probit estimation. Unstandardized coefficients reported. College fixed effects included in all equations. 
Standard errors in parentheses. 

*p < 0.1. **p < 0.05. ***p < 0.01. 
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Table A.8 

Any Non-Zero Points by Metric Per Individual Student: 
Part-Time Students, Academic Year 2011 



Student 

Characteristics 


Basic Skills 


College 

Readiness 


15 Credits 


30 Credits 


Quantitative 

Reasoning 


Degree 

Certification 


Financial aid 


-0.392*** 


0.853*** 


0.543*** 


0.633*** 


0.523*** 


0.521*** 




[0.016] 


[0.009] 


[0.009] 


[0.010] 


[0.010] 


[0.011] 


Age < 20 


0.277*** 


1.839*** 


0.889*** 


1.331*** 


1.226*** 


0.469*** 




[0.027] 


[0.044] 


[0.013] 


[0.022] 


[0.020] 


[0.020] 


Age 20-24 


0.565*** 


1.578*** 


0.298*** 


0.966*** 


0.931*** 


0.748*** 




[0.026] 


[0.043] 


[0.013] 


[0.021] 


[0.020] 


[0.018] 


Age 25-29 


0.695*** 


1.512*** 


0.234*** 


0.926*** 


0.830*** 


0.596*** 




[0.026] 


[0.044] 


[0.014] 


[0.022] 


[0.021] 


[0.019] 


Age 30-34 


0.748*** 


1.413*** 


0.149*** 


0.855*** 


0.712*** 


0.527*** 




[0.026] 


[0.044] 


[0.015] 


[0.023] 


[0.022] 


[0.020] 


Age 34-39 


0.810*** 


1.366*** 


0.150*** 


0.823*** 


0.582*** 


0.496*** 




[0.027] 


[0.045] 


[0.016] 


[0.023] 


[0.023] 


[0.021] 


Age 40-44 


0.849*** 


1.316*** 


0.131*** 


0.789*** 


0.564*** 


0.488*** 




[0.027] 


[0.045] 


[0.017] 


[0.024] 


[0.024] 


[0.022] 


Age 45-49 


0.852*** 


1 279*** 


0.078*** 


0.747*** 


0.492*** 


0.524*** 




[0.028] 


[0.046] 


[0.018] 


[0.025] 


[0.026] 


[0.023] 


Age 50-54 


0.803*** 


1.145*** 


-0.046** 


0.614*** 


0.346*** 


0.391*** 




[0.029] 


[0.047] 


[0.020] 


[0.027] 


[0.028] 


[0.025] 


Age 55-59 


0.693*** 


0.893*** 


-0 191*** 


0.480*** 


0.147*** 


0.352*** 




[0.032] 


[0.051] 


[0.024] 


[0.031] 


[0.034] 


[0.028] 


Age 60-64 


0.578*** 


0.649*** 


-0.405*** 


0.284*** 


-0.105** 


0.043 




[0.038] 


[0.060] 


[0.033] 


[0.039] 


[0.049] 


[0.039] 


Disability 


-0.001 


0.218*** 


-0.021 


0.060*** 


0.060*** 


0.211*** 




[0.024] 


[0.017] 


[0.018] 


[0.018] 


[0.020] 


[0.019] 


Male 


0.006 


0.042*** 


0.015** 


0.045*** 


0.047*** 


0.015* 




[0.008] 


[0.007] 


[0.007] 


[0.007] 


[0.008] 


[0.008] 


Asian 


0.790*** 


0.037*** 


-0.009 


0.001 


0.050*** 


-0.023 




[0.012] 


[0.013] 


[0.012] 


[0.013] 


[0.014] 


[0.015] 


Black 


0.708*** 


0.072*** 


-0.076*** 


-0.090*** 


-0.126*** 


-0.226*** 




[0.015] 


[0.016] 


[0.016] 


[0.017] 


[0.018] 


[0.020] 


Hispanic 


0.804*** 


-0.069*** 


-0.221*** 


-0.211*** 


-0.247*** 


-0.270*** 




[0.011] 


[0.012] 


[0.012] 


[0.013] 


[0.014] 


[0.015] 


Other race 


0.466*** 


0.059*** 


-0.037** 


-0.011 


-0.041** 


-0.067*** 




[0.017] 


[0.015] 


[0.015] 


[0.016] 


[0.017] 


[0.018] 


HS dropout 


0.830*** 


-0.117*** 


-0.082*** 


-0.083*** 


-0.210*** 


-0.044*** 




[0.011] 


[0.012] 


[0.010] 


[0.012] 


[0.012] 


[0.013] 


HS graduate 


0.180*** 


0.384*** 


0.042*** 


0.096*** 


-0.033*** 


0.124*** 




[0.011] 


[0.008] 


[0.008] 


[0.009] 


[0.009] 


[0.010] 


Observations 


343,040 


343,040 


343,040 


343,040 


343,040 


343,040 



Note. Probit estimation. Unstandardized coefficients reported. College fixed effects included in all equations. 
Standard errors in parentheses. 

*p < 0.1. **p < 0.05. ***p < 0.01. 
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Table A.9 

Percentage Distributions Across Start and Follow-up Years 



Metric 


(1) 

Zero Points in Start Year 


(2) 

Basic Skills Points in Start Year 


(3) 

College Readiness Points in Start Year 


(4) 

Credit 15 Points in Start Year 


Basic Skills 


Transfer 


Workforce 


Basic Skills Transfer 


Workforce 


Basic Skills 


Transfer 


Workforce 


Basic Skills 


Transfer 


Workforce 












Points in 2008 for 2007 Start Year 












Zero points 


89 


81 


90 




















Basic skills 


9 


1 


1 


96 


93 


94 














College readiness 


1 


2 


1 


1 


3 


2 


69 


63 


66 








15 credits 


1 


3 


2 


1 


2 


2 


12 


7 


9 


67 


57 


61 


30 credits 


0 


3 


1 


1 


1 


1 


9 


6 


7 


25 


28 


29 


Quantitative reasoning 


0 


4 


1 


1 


1 


1 


8 


15 


11 


6 


14 


6 


Degree completion 


0 


6 


4 


0 


1 


1 


3 


8 


7 


2 


2 


4 












Points in 2010 for 2009 Start Year 












Zero points 


89 


82 


87 




















Basic skills 


9 


0 


1 


96 


92 


92 














College readiness 


1 


2 


1 


2 


3 


2 


73 


78 


64 








15 credits 


1 


3 


3 


1 


2 


3 


10 


7 


8 


66 


58 


61 


30 credits 


0 


2 


2 


1 


1 


2 


8 


6 


10 


25 


27 


28 


Quantitative reasoning 


0 


3 


1 


0 


1 


1 


8 


1 


11 


6 


13 


6 


Degree completion 


0 


7 


5 


0 


0 


1 


2 


7 


7 


4 


2 


5 












Points in 2011 for 2010 Start Year 












Zero points 


89 


82 


87 




















Basic skills 


8 


0 


1 


96 


86 


93 














College readiness 


1 


2 


1 


1 


4 


2 


66 


65 


64 








15 credits 


1 


3 


2 


1 


4 


2 


9 


8 


9 


64 


54 


59 


30 credits 


0 


2 


2 


1 


2 


1 


12 


7 


10 


24 


30 


28 


Quantitative reasoning 


0 


3 


1 


1 


2 


1 


10 


13 


12 


9 


14 


8 


Degree completion 


0 


7 


6 


0 


1 


1 


2 


7 


6 


4 


2 


5 



Note. Numbers rounded to nearest percentage. 
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Table A.10 

Percentage Distributions Across Start and Follow-up Years: 

Transfer and Workforce Students Who Attempted Either English or Math Remedial Courses 



Metric 


(1) 

Zero Points 
in Start Year 


(2) 

Basic Skills Points 
in Start Year 


(3) 

College Readiness Points 
in Start Year 


(4) 

Credit 15 Points 
in Start Year 


Transfer 


Workforce Transfer 


Workforce 


Transfer 


Workforce 


Transfer 


Workforce 








Points in 2008 for 2007 Start Year 








Zero points 


85 


90 














Basic skills 


0 


0 


67 


74 










College readiness 


5 


1 


19 


7 


63 


66 






15 credits 


3 


3 


3 


4 


7 


9 


47 


48 


30 credits 


2 


2 


3 


6 


6 


7 


30 


33 


Quantitative reasoning 


3 


2 


8 


2 


15 


11 


21 


15 


Degree completion 


2 


2 


0 


7 


8 


7 


2 


4 








Points in 2010 for 2009 Start Year 








Zero points 


83 


84 














Basic skills 


0 


0 


55 


85 










College readiness 


6 


5 


7 


6 


67 


64 






15 credits 


3 


3 


17 


7 


7 


8 


47 


46 


30 credits 


2 


3 


14 


2 


6 


10 


31 


33 


Quantitative reasoning 


3 


2 


7 


0 


13 


11 


20 


15 


Degree completion 


2 


2 


0 


0 


7 


7 


2 


6 








Points in 2011 for 2010 Start Year 








Zero points 


86 


88 














Basic skills 


0 


0 


77 


74 










College readiness 


6 


4 


8 


11 


65 


74 






15 credits 


3 


3 


12 


6 


8 


9 


45 


44 


30 credits 


2 


2 


0 


6 


7 


10 


32 


35 


Quantitative reasoning 


2 


2 


4 


2 


13 


1 


21 


16 


Degree completion 


2 


1 


0 


1 


7 


6 


2 


5 



Note. Numbers rounded to nearest percentage. Sample sizes for college ready students < 100. 
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